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ABSTRACT 


This  thesis  develops  and  analyzes  a  model  to  identify 
attributes  of  a  successful  recruiter.  Expert  Systems 
software  is  used  to  elicit  from  twenty  U.S.  Army  recruiters 
and  instructors,  who  are  experts  in  the  field, 
characteristics  associated  with  recruiter  success.  An 
interactive  computer  program  based  on  Quasi-Artificial 
Intelligence  (QAI)  captured  the  expert's  intuition, 
knowledge,  experience,  and  judgments  to  create  expert 
systems  that  can  be  usee  to  select  U.S.  Army  recruiters 
before  they  attend  recruiting  school.  This  study  found  that 
personal  characteristics  such  as  Integrity  and  Commitment, 
and  skills  such  as  Persuading  and  Listening  are 
substantially  more  important  than  the  types  of  attributes 
generally  used  to  predict  recruiter  success. 
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I.  INTRODUCTION 


A.  OVERVIEW 

i.  The.  Prpfrlsrc 

In  the  years  ahead,  many  factors  could  threaten  the 
continuation  of  the  All-Volunteer  armed  forces  (AVF) . 
Recruiting  is  expected  to  become  more  difficult  as  a 
declining  youth  population  decreases  the  pool  of  potential 
recruits.  In  addition,  in  times  of  prosperity,  the  military 
will  continue  to  compete  with  perhaps  more  attractive 
civilian  employment.  Finally,  budget  constraints  may  make 
it  even  more  difficult  to  expend  the  resources  necessary  to 
attract  the  desired  quantity  and  quality  of  recruits. 

The  services  will  increasingly  need  to  maximize  the 
efficient  utilization  of  its  manpower.  An  area  of  prime 
consideration  and  importance  is  recruiter  selection. 
Recruiting  problems  are  aggravated  when  the  wrong 
individuals  are  sent  to  recruiting  assignments.  The  costs 
to  the  military  are  considerable  in  terms  of  both  monetary 
and  human  resources.  For  example,  one  Navy  study  involving 
1,262  male,  active  duty  canvasser  recruiters  in  1979 
revealed  a  drop-out  rate  of  19  percent,  costing  almost  three 
million  dollars  in  base  pay,  BAQ ,  and  PCS  costs  alone. 

[Ref.  1 : p.  68] 


5 


In  addition,  the  recruiting  commands  suffer  because 
of  losses  in  productivity,  and  the  military  as  a  whole 
suffers  because  of  the  loss  of  petty  officer  talents  in 
positions  throughout  the  operating  forces.  The  military  is 
not  the  sole  loser,  however.  Recruiter  selection  is  based 
on  good  performance  as  well  as  other  criteria.  The 
individuals  selected  for  recruiting  duty  are  usually  some  of 
the  best  the  services  have  to  offer  with  respect  to  previous 
assignments.  If  these  "successful"  senior  enlisted 
personnel  are  not  successful  on  recruiting  duty,  their 
self-confidence,  attitude  and  motivation  will  probably 
suffer  with  their  loss  of  productivity  and  affect  future 
assignments,  or  cause  them  to  get  out  of  the  service  earlier 
than  anticipated. 

Although  improved  recruiter  selection  will  not  solve 
all  the  services  recruiting  problems,  it  could  increase 
productivity  and  morale  because  of  better  recruiter/ job 
matches,  and  reduce  turnover  and  related  costs  from  moving, 
training,  and  replacing  recruiters  who  are  not  right  for  the 
job.  This  thesis  will  assist  recruiter  selection  by 
developing  a  tool  that  may  be  useful  for  reducing  the 
selection  of  personnel  that  do  not  become  successful 
recruiters. 

2 .  Literature  Review 

An  in-depth  literature  review  was  conducted  in  a 
prior  masters  thesis  at  the  Naval  Postgraduate  School  by 
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LCDR  Joyc«  Zellweger  on  the  selection  of  successful 
recruiters  [Ref.  2].  My  review  will  summarize  her  research, 
emphasize  the  studies  most  relevant  to  identifying 
successful  recruiters,  and  identify  strengths  and  weaknesses 
of  each  approach.  In  addition,  an  annotated  bibliography  is 
provided  in  Appendix  A  to  this  thesis. 

As  the  following  summary  will  show,  although  most 
prior  studies  presented  reasonable  hypotheses,  sound 
analysis,  and  interesting  conclusions,  few  of  the  findings 
were  significant.  The  findings  that  were  significant  had 
questionable  reliability  because  they  were  not  cross- 
validated.  In  others,  when  cross-validation  was  attempted, 
original  results  could  not  be  duplicated.  A  common  problem 
in  prior  studies  was  the  "criterion  problem" — measuring 
recruiter  performance  in  a  reliable  and  valid  manner. 
Researchers  have  used  a  variety  of  different  performance 
measures,  including  supervisory  ratings,  school  performance, 
percent  of  quota  achieved,  and  total  number  of  recruits 
enlisted.  Yet,  many  of  these  measures  are  often  unreliable 
and  of  questionable  validity.  For  example,  supervisory 
ratings  are  often  based  on  the  individual's  reputation 
rather  than  performance.  "Even  with  the  best  of  intentions, 
supervisors  can  be  influenced  by  characteristics  unrelated 
to  job  effectiveness."  [Ref.  2:pp.  18-19]  Other  problems 
researchers  encountered  were  lack  of  information  about  the 
recruiter's  job,  and  reliable  recruiter  selection  methods 
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(since  most  active  duty  recruiters  are  selected 

involuntarily).  [Ref.  2:p.  20]  _  _ _ 

The  following  studies  are  organized  by  the  source  of 
information  used  to  identify  successful  recruiters: 
interviews,  test  batteries,  assessment  centers,  and 
personnel  file  data. 

a.  Interviews 

The  studies  involving  interviews  provided 
little  empirical  evidence  to  test  hypotheses  pertaining  to 
successful  recruiter  attributes.  However,  they  did  provide 
a  springboard  for  additional  studies  and  identified  the 
types  of  personal  characteristics  and  attributes  necessary 
for  effective  recruiter  performance.  In  most  cases, 
however,  the  data  obtained  were  based  on  opinions,  involved 
a  biased  sample,  or  were  simply  a  framework  for  further 
research. 

Some  of  the  attributes  of  successful 
recruiters  identified  are: 

-  They  are  "movers,"  "shakers,"  and  salesmen; 

-  They  are  hungry  for  success  and/or  promotion; 

-  They  are  aggressive,  want  responsibility,  and  want  to 
excel ; 

-  They  can  communicate  effectively; 

-  They  are  friendly,  easygoing,  outgoing,  sympathetic, 
stable,  and  sincere; 

-  They  are  ambitious,  extroverted,  and  self-motivated. 
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b.  Test  Batteries 


Most  of  the  studies  involving  test  batteries 
yielded  disappointing  results  primarily  because  they  were 
not  cross-validated  or  simply  could  not  measure  recruiter 
performance  ir.  a  reliable  and  valid  manner.  A  study 
developed  by  the  Navy  Personnel  Research  and  Development 
Center  (NPRDC)  (Borman,  et  al.,  1976-1986)  provides  the  most 
extensive  and  promising  work  in  this  area.  This  work  has 
evolved  through  four  studies  over  the  past  ten  years. 

NPRDC  began  with  the  development  of 
behaviorally-based  rating  scales  which  attempted  to  identify 
improved  performance  criteria  for  measuring  recruiter 
effectiveness.  This  first  study  (Borman,  et  al.,  1976) 
identified  more  than  800  critical  incidents  describing 
different  facets  of  effective  and  ineffective  recruiter 
performance  (see  Table  1).  The  second  (Borman,  et  al., 

1976)  phase  involved  development  and  validation  of  an 
inventory  battery  to  predict  Navy  and  Marine  Corps  recruiter 
performance.  They  developed  a  trial  predictor  battery  that 
included  several  personality,  vocational  interest,  and 
biographical  items  and  scales.  In  the  third  phase  (Borman 
et  al.,  1981),  the  original  test  battery  was  expanded  and 
refined.  The  revised  battery  was  analyzed  to  determine  the 
precision  of  new  items  in  measuring  desired  constructs  and 
whether  they  had  improved  the  validity  of  the  original  test 
battery.  The  added  items  did,  in  fact,  enhance  the  validity 
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of  the  old  battery's  constructs  in  about  half  the  cases. 
Scales  derived  from  these  constructs  validly  predicted 
recruiter  effectiveness. 

NPRDC's  final  Special  Assignment  Battery 
measured  recruiter  potential  through  a  selection  composite 
of  four  sub-scales:  selling  skills,  human  relations  skills 
organizing  skills,  and  overall  performance.  Scores  in  each 
of  these  areas  were  correlated  with  each  recruiter's 
production  data.  Organizing  skills  was  the  only  sub-scale 
that  was  statistically  insignificant.  - - — 

Several  personality  constructs  correlated 
highly  with  various  aspects  of  recruiter  effectiveness: 

-  "Making  a  good  impression"  and  "Enjoying  being  the 
center  of  attention"  correlated  highest  with 
selling  skills. 

-  "Spontaneity,  impulsiveness,  ambitious,  working  hard" 
correlated  highest  with  the  human  relations  skills 
category. 

-  "Unhappy,  lack  of  confidence"  related  negatively  to 
human  relations  skills  effectiveness. 

-  "Order,  planning  ahead"  related  well  to  organizing 
skills. 

-  "Leading  and  influencing  others"  was  the  construct 
_ Jthat^  correlated  most  highly  in  the  Overall  performance 

category. 

-  Interests  in  extroverted,  dominant,  social,  and 
leadership  activities  and  occupations,  interests  in 
sports  and  competitive  activities,  and  interests  in  law 
and  political  activities  correlated  highly  with 
vocational  interests. 

The  fourth  phase  of  NPRDC's  work,  published 
in  1985  (Borman,  et  al.,  1986),  strongly  confirmed  the 
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findings  of  the  earlier  studies.  In  concurrent  studies, 
Marine  Corps  recruiters  whose  scores  were  in  the  top  20 
percent  obtained  27  percent  more  recruits  than  recruiters  in 
the  lowest  20  percent.  In  predictive  studies,  they  obtained 
40  percent  more  recruits. 

c.  Assessment  Centers 

The  assessment  center  concept  involved  using 
trained  observers  to  rate  a  potential  recruiter’s 
performance  during  exercises  simulating  aspects  of  the 
recruiter  job.  Assessors  focused  on  personal 
characteristics  such  as  persuasiveness,  sociability, 
flexibility,  and  practical  judgment.  The  results  were 
successful  in  predicting  recruiter  school  performance, 
however  the  concept  was  based  on  the  assumption  that  the 
people  being  rated  wanted  the  job.  Since  most  recruiters 
are  assigned  involuntarily,  the  assessment  center  concept 
proved  to  be  infeasible. 

In  1985  a  study  was  conducted  by  Weltin, 
et  al.,  attempting  to  evaluate  the  usefulness  of  the  ratings 
from  the  original  assessment  center  for  predicting  job 
performance  as  a  field  recruiter.  Each  individual  had  been 
rated  by  trained  assessors  in  exercises  including  cold 
calls,  interviews,  a  speech,  and  an  in-basket  (work 
prioritization) .  The  assessment  center  sample  included  41 
of  57  soldiers  who  had  taken  the  original  battery  in  1981 
and  completed  the  training  course.  Concurrently,  a  similar 
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study  was  conducted  on  a  development  center  sample  of  970 
recruiters  who  were  rated  in  the  center,  completed  training, 
and  had  at  least  one  contract  their  first  year  on  the  job. 
However,  assessors  were  not  trained  and  instructor  ratings 
on  telephone  and  interviewing  techniques  were  not  available. 

The  results  of  the  assessment  center  ratings 
showed  low  correlations  with  job  performance.  Furthermore, 
in  the  development  center  sample,  only  the  cold  call 
interview  and  speech  exercises  were  significant.  A  stepwise 
regression  was  performed  on  the  development  center  sample, 
using  job  performance  as  the  dependent  variable,  with 
ratings,  training  grades,  and  other  predictors  used  as  the 
independent  variables.  The  results  indicated  that 
productivity  of  the  recruiter's  battalion  was  the  single 
most  important  factor  in  predicting  job  performance, 
d.  Personnel  File  Variables 

The  studies  in  this  area  also  revealed 
attributes  that  successful  recruiters  possess.  However, 
most  were  not  cross-validated  or  failed  in  cross-validation 
and  therefore  have  limited  usefulness. 

The  best  study  was  conducted  in  1983  by  Elig, 
et  al.,  which  used  the  Enlisted  Master  File  (EMF)  and  the 
Military  Enlistment  Processing  Station  Reporting  System 
(MRS)  as  their  data  source  to  acquire  information  on  recruit 
characteristics.  The  sample  consisted  of  552  male  and  60 
female  Army  recruiters  on  recruiting  duty  during  fiscal  year 
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1979.  Descriptive  statistics  were  used  to  analyze 
characteristics  of  recruiters.  Those  characteristics  which 
correlated  with  contract  production  were  identified  using 
analysis  of  covariance  techniques.  Results  indicated  that 
opportunity  bias  (District  Recruiting  Command's  Average 
Production)  explained  32  percent  of  the  variance  in 
productivity.  The  remaining  variance  was  believed  to  have 
been  a  result  of  unmeasured  opportunity  bias.  Some  of  their 
findings,  significant  to  at  least  the  .01  level  were  as 
follows: 

-  Recruiters  with  post-secondary  education  recruited 
better  educated,  but  lower  AFQT,  males. 

-  Recruiter  AFQT ' s  correlated  positively  with  recruit 
AFQT’s  in  its  "prime"  market  (high  school  diploma 
graduate  and  senior  males) ,  but  had  little  impact  on 
females  or  non-high  school  graduates. 

-  Recruiter  gender  had  no  effect. 

-  Older  recruiters  contracted  more  male  and  fewer  female 
recruits  than  younger  recruiters. 

-  In  total  production,  younger  males  out-performed  older 
males  and  their  female  counterparts,  while  older  females 
out-performed  all  others. 

-  Higher  ranking  recruiters  achieved  success  in  the  high 
school  diploma  graduates  and  senior  males  market  by 
contracting  more  low  AFQT  (category  IV)  recruits  than 
lower  ranking  recruiters. 

-  Black  recruiters  enlisted  the  most  Blacks,  Hispanic 
recruiters  enlisted  the  most  Hispanics,  whites  the  most 
whites,  etc. 

e.  Summary 

Most  of  the  past  research  on  recruiter 
selection  suffered  from  poor  criterion  measurement,  lack  of 
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knowledge  of  the  recruiter  job,  or  failure  of  results  to 
remain  significant  upon  cross-validation.  As  a  result, 
findings  of  many  of  these  studies  are  of  questionable  value. 

Encouraging  results  in  recruiter  selection 
research  was  found  in  the  study  conducted  by  NPRDC  on  the 
Special  Assignment  Battery  and  the  assessment  center 
concept.  However,  both  methods  are  very  costly,  which 
reduces  their  potential  for  application. 

The  Special  Assessment  Battery  proved  to  be 
highly  valid  when  cross-validated  on  a  sample  of  Marine 
Corps  recruiters.  The  characteristics  which  appeared  to  be 
associated  with  recruiter  success  were  background  and 
personality  characteristics,  and  interest  patterns. 

However,  the  battery  is  very  lengthy  and  costly  to 
administer.  Furthermore,  if  future  non-volunteer  recruiters 
who  took  the  battery  believed  their  scores  would  result  in  a 
recruiting  assignment,  potential  sabotage  could  reduce  the 
tool’s  usefulness.  To  get  around  this,  personnel  could  be 
required  to  complete  the  battery  well  before  a  time  when 
they  would  associate  it  with  recruiting  duty,  but  these 
would  impose  even  greater  costs  on  the  military.  [Ref.  2: 
p.  64] 

The  assessment  center  concept  is  used  in  the 
military  as  a  part  of  recruiter  training  to  indoctrinate  and 
familiarize  recruiters  with  what  to  expect  on  their  way  to 
the  field.  But  assessment  centers  are  also  costly.  Those 
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who  do  not  complete  the  training  are  transferred  to  other 
jobs,  while  the  job  in  the  field  is  left  empty  until  another 
person  can  be  identified  and  complete  the  training.  In 
addition  to  wasted  transfer  dollars  caused  by  the 
unnecessary  moves,  other  types  of  hardships  may  arise  for 
the  member  or  his/her  family.  [Ref.  2:pp.  64-65] 

Several  studies  attempted  to  identify  passive 
recruiter  selection  procedures  to  identify  people  who  would 
most  likely  become  successful  recruiters  before  being 
assigned  to  recruiting  duty.  Some  of  the  personal  and 
background  characteristics  were  statistically  significant, 
but  the  relative  importance  of  these  characteristics  in 

recruiter  selection  could  not  be  determined. _ Budget  cuts  _ 

and  increasing  numbers  of  non-volunteer  recruiters  make 
passive  selection  procedures  more  important,  but  significant 
research  questions  remain  unanswered.  [Ref.  2:p.  65] 
f.  Purpose  of  Thesis 

Characteristics  believed  to  be  related  to 
successful  recruiting  have  been  identified  and  summarized  in 
Table  2,  but  which  cf  these  characteristics  is  most 
important?  How  many  prospective  recruiters  possess  all  of 
the  characteristics  believed  to  be  part  of  the  successful 
recruiter  profile?  If  one  person  has  some  of  these 
characteristics,  and  another  person  has  some  of  the  others, 
who  do  we  choose?  [Ref.  2:pp.  65-66] 
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TABLE  2 


SUMMARY  OF  CHARACTERISTICS  RELATED  TO  RECRUITER  SUCCESS 

Plans  Ahead 

Uses  Systematic  Approach  in  Prospecting 
Knowledgeable  About  Recruiting 
Sales  Experience 

Age  (older  if  female,  younger  if  male) 

Marital  Status 
Paygrade 

Length  of  Service 
AFQT  Scores 

Verbal  Fluency 
Persuasiveness 
Communicates  Effectively 

Self-Motivated 
Ambitious 
Desire  to  Excel 
Aggressive 
Dominant 
Confident 
Enthusiastic 
Positive 
Mature 

Financially  Stable 
Extroverted 

Enjoys  Working  with  Others 
Spontaneous 
Influences  Others 

Well  Groomed 


The  failure  of  empirical  analysis  to 
successfully  predict  the  kind  of  person  who  will  become  a 
successful  recruiter  may  stem  from  an  inability  to  invoke 
heuristics  into  "conventional"  computer  systems.  Heuristics 
involve  hunches,  educated  guesses,  and  rules  of  thumb  which 
are  based  on  experience  and  knowledge  of  underlying 
principles.  Perhaps  the  missing  link  to  understanding  the 
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profile  of  a  successful  recruiter  is  the  judgement  of  an 
expert  who  has  worked  in  this  area  and  knows  better  than 
anyone,  innately,  what  it  takes  to  be  a  successful 
recruiter. 

Therefore,  this  thesis  will  attempt  to  develop 
a  tool  to  assist  in  recruiter  selection  that  may  be  useful 
in  identifying  individuals  with  high  likelihood  of  being 
successful  recruiters. 

The  next  chapter  describes  a  methodology  for 
making  decisions  about  the  relative  importance  of  the 
characteristics  believed  to  be  important  in  the  profile  of 
successful  recruiter.  Chapter  III  will  analyze  the 
similarities  and  differences  of  the  expert  systems  created 
by  each  of  the  experts.  Chapter  IV  will  then  summarize  my 
findings  and  provide  recommendations  for  future  work  in 
recruiter  selection. 


II.  METHPPQLQgY 


The  goal  of  this  thesis  is  to  identify  the  most 
important  factors  and  their  relative  importance  for 
identifying  individuals  with  high  likelihood  of  being 
successful  recruiters.  To  accomplish  this,  I  am  going  to 
improve  the  present  recruiter  model  and  apply  expert  systems 
analysis  to  derive  a  recruiter  selection  model  for  reserve 
Army  recruiters.  Previous  attempts  to  assess  characteris¬ 
tics  that  can  be  used  to  reliably  select  successful 
recruiters  has  suffered  from  the  deficiencies  indicated  in 
the  last  chapter.  The  expert  systems  approach  will  attempt 
to  "fill  in  the  gaps"  of  previous  studies  and  provide  the 
military  with  a  tool  to  improve  recruiter  selection 
procedures. 

A.  ARTIFICIAL  INTELLIGENCE  AND  EXPERT  SYSTEMS 
1.  Background  and  Definition 

The  science  of  Artificial  Intelligence  (AI)  has 
grown  significantly  in  the  past  few  years,  particularly  in 
commercial  applications.  It  began  in  the  1950s  evolving 
from  the  fields  of  mathematics  and  philosophy  and  the  desire 
of  cognitive  psychologists  to  understand  human  memory  and 
reasoning.  The  first  computers  were  able  to  formalize  and 
emulate  many  procedural  activities.  Since  intelligence 
involves  manipulation  of  symbols,  much  of  AI  is  based  not  on 
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calculating  optimal  answers,  but  on  the  use  of  heuristics 
that  can  provide  quick,  satisfactory  answers.  [Ref.  3] 

There  are  many  definitions  of  AI.  A  scientist  or 
research  specialist  may  define  it  as  the  study  of  ideas 
which  enable  computers  to  do  things  that  make  people  seem 
intelligent.  A  more  goal-oriented  definition  is  making 
machines  "smart"  so  they  will  become  more  useful  and 
understand  the  principles  which  make  intelligence  possible. 
An  applications  definition  is  a  field  of  computer  problem 
solving.  A  useful  definition  for  current  purposes  is  an 
effort  to  develop  computer  systems  that  can  approximate  a 
human's  ability  to  reason  and  decide.  [Ref.  4] 

All  areas  of  AI  are  involved  in  the  extraction  or 
generation  of  information  and  in  understanding  the 
surrounding  environment  (complex  set  of  data  or  goals) .  One 
of  the  most  significant  efforts  in  AI  has  been  in  the  area 
of  expert  or  knowledge-based  systems.  Expert  systems  model 
complex  situations  in  one  specific  domain  and  provide 
conclusions  based  on  a  set  of  rules  that  have  been  reduced 
from  the  knowledge  and  experience  of  people  who  have 
expertise  in  a  given  functional  area  (domain  experts) 

[Ref.  5].  Knowledge  engineers  attempt  to  elicit  and 
formalize  problem  domain  information  from  human  experts  in 
the  field  through  interviews  and  test  cases.  The  knowledge 
engineer  becomes  acquainted  with  the  facts,  identifies  the 
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concepts,  and  develops  and  codifies  the  heuristics  that  the 
expert  steps  through  while  solving  a  problem  [Ref.  4 j . 

Expert  systems  attempt  to  provide  an  answer  to  the 
age-old  problem  of  knowledge  transfer.  That  is,  while  still 
lacking  many  of  the  characteristics  of  true  human  expertise, 
expert  system  technology  may  make  the  skills  of  an  expert 
available  to  a  broad  population  who  are  not  experts.  (An 
expert  is  someone  who  has  developed  more  knowledge  in  a 
particular  subject  area  than  most  people  in  the  same  field, 
and  who  can  use  that  knowledge  to  work  more  efficiently  and 
effectively.)  In  addition,  once  the  expert  system  is 
developed,  technicians  can  maintain  it  without  being  experts 
themselves.  This  is  particularly  useful  because  the 
know-how  developed  over  years  of  experience  and  concentrated 
effort  will  not  be  lost  when  the  expert  dies  [Ref.  6], 

As  shown  in  Figure  1,  the  basic  parts  of  an  expert 
system  are: 

a.  Knowledge  Base.  The  most  common  method  of  represent¬ 
ing  knowledge  is  through  the  rules  about  a  specific 
domain  in  the  form  of  facts  and  heuristic  rules. 

Facts  are  known  rules  about  the  domain  of  information, 
while  heuristics  are  those  rules  of  good/ber.ter 
judgment  developed  through  experience  and  trial-and- 
error  methods  by  domain  experts  while  solving  problems 
[Ref.  5].  The  knowledge  base  also  includes  some  of 
the  system's  programs.  These  programs  manipulate  the 
symbols  which  represent  the  facts  and  rules  of  the 
knowledge  domain.  The  computer  follows  a  few  simple 
procedures,  such  as  searching,  matching,  separating, 
joining,  substituting,  and  deleting  when  processing 
the  data  or  symbols  in  order  to  find  a  solution  to  the 
problem.  This  is  where  AI  programming  is  so  differ¬ 
ent  from  the  conventional  program  approach  of  follow¬ 
ing  specific  step-by-step  procedures.  In  AI,  the 
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Figure  1.  Components  of  an  Expert  System 


distinction  between  the  facts  (data)  and  the  program 
is  blurred  (Ref.  4). 

b.  Inference  Engine.  The  program  which  controls  the 
evaluation  of  a  problem  and  evaluates  the  rules  in  the 
knowledge  base  to  define  a  solution  [Ref.  5]. 

c.  Language  Processor.  Allows  a  user  to  interface  with 
the  system  to  enter  or  retrieve  data  [Ref.  4],  It 
parses  and  interprets  questions  from  the  user  and  then 
formats  the  question  for  the  inference  engine. 

Finally,  when  an  answer  is  reached,  the  language 
processor  formats  the  information  for  the  user  [Ref. 
5]. 

d.  Knowledge  Acquisitioner.  The  person  who  ensures  that 
the  knowledge  base  has  correct  and  current  information 
and  that  the  rules  work  together  properly  [Ref.  5]. 
Knowledge  acquisition  is  very  difficult  because  human 
knowledge  is  complex  and  can  be  messy  and  ill- 
formulated;  humans  find  it  extremely  difficult  to 
articulate  the  knowledge  they  have  and  how  they  use 
that  knowledge  to  solve  problems;  the  more  expert  a 
person  becomes,  the  more  'unconscious'  his/her 
knowledge  becomes;  the  data  from  knowledge  acquisition 
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techniques  need  careful,  even  skilled,  interpretation 
as  to  what  underlying  knowledge  may  be  inferred  from 
them;  and  the  techniques  which  have  been  developed  are 
still  only  poorly  understood,  not  very  robust,  and 
often  have  very  limited  applicability  [Ref.  7}. 

Once  the  knowledge  engineer  has  collected  and 

assimilated  the  data,  he/she  must  choose  from  among  several 

methods  how  best  to  represent  the  expert's  knowledge  in  the 

system.  Knowledge  representation,  or  representing  facts  and 

rules  about  a  knowledge  domain,  can  be  accomplished  through 

one  or  a  combination  of  approaches.  Ttrae  of  the  most 

common  techniques  are  Semantic  Networks,  Frames,  and 

Production  Rules.  _ _ _ 

Semantic  Networks  were  originally  proposed  in  1966 

as  a  model  of  the  human  memory.  They  were  devised  to 

represent  relationships  among  various  data  elements  that  are 

found  in  memory.  A  common  representation  scheme  consists  of 

a  tree-like  structure  of  nodes  which  represent  concepts  and 

links  or  branches  between  the  nodes  which,  by  being  linked, 

represent  attributes  that  can  be  "inherited"  down  the  link. 

The  best  characterization  of  these  links  is  the  "is-a" 

statement.  For  example,  a  blue- jay  is-a  bird,  a  bird  is-a 

mammal.  This  type  of  structure  leaves  room  for  expansion  of 

inheritance  characteristics  or  attributes  at  each  node.  The 

links  containing  the  attributes  are  called  by  a  variety  of 

names  such  as  has-a,  part-of,  or  has-part.  Thus,  attributes 

can  be  inferred  from  nodes  to  which  they  are  linked.  An 

example  illustrating  this  idea  is  a  bird  has-a  wing,  a 
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blue-jay  is-a  bird,  therefore,  a  blue-jay  has-a  wing.  The 
advantage  of  this  technique  lies  in  its  excellent  ability  to 
portray  relationships. 

The  Frames  technique  is  basically  a  data  structure 
similar  to  a  template  with  "slots"  reserved  for  different 
attributes,  and  the  frame  description  has  all  the  basic 
information  including  relationships.  A  "cat"  frame  could 
include  a  description  of  status  (pet) ,  name  ("Tinker") , 
breed  (Siamese) ,  and  color  (brown  and  white) .  Another 
person  would  also  have  a  frame  with  the  same  generic 
information.  Both  frames  would  be  linked  to  the  conceptual 
"cat"  frame  where  various  attributes  such  as  name,  breed, 
and  physical  attributes  are  contained  in  "slots".  Each  slot 
specifies  a  value  or  range  of  values  for  each  attribute. 

This  technique  represents  objects  by  standard  attributes  and 
relationships  to  other  frames.  The  advantage  of  this 
technique  is  in  the  amount  of  knowledge  that  can  be  stored 
about  the  attributes  and  relationships  of  the  object  in 
question. 

The  Production  Rules  technique  is  the  most  popular 
in  building  expert  systems.  It  was  developed  in  1972  as  a 
model  for  human  cognition.  The  knowledge  base  consists  of 
an  IF  statement  which  spells  out  a  pattern  or  condition,  and 
an  action  part  designated  by  a  THEN  statement  which 
elaborates  an  action  to  be  taken  or  a  consequence  once  the 
condition  has  been  satisfied.  For  example,  IF  the  cruise 
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control  is  disengaged  and/or  the  accelerator  is  released, 
THEN  the  car  will  slow  down.  The  condition  can  be  refined 
by  adding  more  qualifications,  which  makes  it  particularly 
appealing  for  use  in  a  military  expert  system  since  the 
rules  can  be  very  explicit.  The  advantage  of  this  technique 
is  its  ability  to  represent  procedural  knowledge. 

The  most  difficult  part  of  this  whole  process  is  to 
capture  the  human  thought  process.  The  human  mind  can 
understand  statements  which  are  merely  inferred  and 
appropriately  weigh  or  associate  incoming  information  with 
past  experience.  But,  how  can  this  be  programmed  into  a 
computer?  One  of  the  methods  used  to  weigh  confidence 
levels  in  expert  systems  is  the  assignment  of  confidence 
factors  which  are  usually  attached  to  the  rules.  The 
reasoning  mechanism  used  in  the  inference  engine  is  chosen 
by  the  knowledge  engineer  and  is  usually  of  two  types: 
forward  chaining  or  backward  chaining.  _ 

Forward  chaining  attempts  to  achieve  a  goal  given  an 
initial  state.  It  is  said  to  be  data  driven.  Backward 
chaining  works  backward  from  a  hypothesis  to  seek  the 
evidence  that  will  support  it.  It  is  said  to  be  goal  driven 
[Ref.  4].  Forward  chaining  seeks  to  identify  all  rules 
whose  IF  portions  are  true,  then  uses  the  THEN  portions  of 
those  rules  to  find  other  rules  which  are  also  true.  In 
other  words,  when  trying  to  solve  a  problem,  this  method 
takes  a  kind  of  trial  and  error  approach  through  the 
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information — both  permanent  facts  and  rules  that  have  been 
built  in,  and  facts  supplied  by  the  person  using  the  system 
pertaining  to  the  problem  at  hand. 

On  the  other  hand,  backward  chaining  seeks  to 
satisfy  a  stated  goal  by  seeking  rules  in  which  the  THEN 
portion  matches  the  goal,  then  seeks  other  rules  whose  THEN 
portion  matches  the  IF  portion  of  the  rule  that  it 
satisfies.  It  starts  from  the  desired  result  and  works 
backward  through  the  rules  in  its  possession,  and  considers 
only  those  that  are  relevant  to  the  goal  [Ref.  5]. 

2.  Ev?lyti9n..gf  an  Expert  .System 

a.  Identification  (Determining  Problem 
Characteristics) 

The  expert  and  the  knowledge  engineer  work 
together  to  identify  the  problem  and  define  its  scope.  They 
also  identify  the  participants  in  the  development  process 
(additional  experts) ,  determine  the  resources  needed  and 
decide  on  the  goals  and  objectives  of  building  the  expert 
system. 

b.  Conceptualization  (Finding  Concepts  or  Represent 
Knowledge) 

The  expert  and  the  knowledge  engineer  explicate 
the  xey  concepts,  relationships,  and  information-flow 
characteristics  needed  to  describe  the  problem-solving 
process  in  the  given  domain.  They  also  specify  sub-tasks, 
strategies,  and  constraints  related  to  the  problem-solving 
activity. 


27 


c.  Formalization  (Designing  Structures  to  . . 

Organizational  Knowledge) 

The  knowledge  engineer  must  select  the  language, 
and  with  the  help  of  the  expert,  represent  the  basic 
concepts  and  relationships  within  the  language  framework. 

In  other  words,  map  the  key  concepts  into  a  formal 
representation. 

d.  Implementation  (Formulating  Rules  that  Embody 

Knowledge) 

The  knowledge  engineer  combines  and  reorganizes 
the  formalized  knowledge  to  make  it  compatible  with  the 
information  flow  characteristics  of  the  problem.  The 
resulting  set  of  rules  and  control  structures  associated 
with  them  define  a  prototype  program  which  is  capable  of 
being  executed  and  tested. 

e.  Testing  (Validating  Rules  that  Embody 

Knowledge) 

The  performance  of  the  prototype  program  must 
then  be  evaluated  and  revised  to  conform  to  the  standards  of 
excellence  defined  by  the  experts  in  the  problem  domain. 

The  most  important  goal  in  expert  systems 1  work 
is  to  attain  the  high  level  of  performance  that  a  human 
expert  would  achieve  in  the  same  task.  The  quality  of 
reasoning  in  an  expert  system  is  based  on  the  accessibility 
of  relevant  facts  and  principles  as  well  as  on  completeness 
of  the  inference  procedure  and  efficacy  of  its 
implementation.  One  way  to  determine  if  the  expert  system 
is  successful  is  to  examine  its  ability  to  reason  about  its 
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own  processes  and  be  able  to  satisfy  another  expert  as  to 
the  soundness  of  the  reasoning  sequence.  Explanation  of 
this  process  is  usually  associated  with  some  form  of  tracing 
of  rules  that  are  used  during  the  course  of  a  problem¬ 
solving  session,  and  reconstructing  a  rational  line  of 
argumenc  built  out  of  the  fundamental  principles  of  the 
domain  [Ref.  7]. 

3.  Types  of  Expert  Systems 

Most  expert  system  applications  fall  into  a  few 
distinct  types.  The  following  generic  categories  will 
briefly  describe  the  problem  each  addresses: 

a.  Interpretation — Infers  situation  descriptions  from 
sensor  data. 

b.  Prediction — Infers  likely  consequences  of  given 
situations . 

c.  Diagnosis — Infers  system  malfunctions  from 
observables . 

d.  Design — Configuring  objects  under  constraints. 

e.  Planning — Designing  actions. 

f.  Monitoring — Comparing  observations  to  plan 
vulnerabilities. 

g.  Debugging — Prescribing  remedies  for  malfunctions. 

h.  Repair — Executing  a  plan  to  administer  a  prescribed 
remedy. 

i.  Instruction — Diagnosing,  debugging,  and  repairing 
student  ber.avior. 

j.  Control — Interpreting,  predicting,  repairing,  and 
monitoring  system  behaviors.  [Ref.  8] 
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The  following  section  will  present  some  examples  of 
commercial  and  military  applications  of  expert  systems 
utilizing  some  of  these  types. 

4.  Applications  of  Expert  Systems 
a.  Commercial 

(1)  Leukemia  Diagnosis.  An  expert  system  was 
developed  to  interpret  the  results  of  laboratory  tests  in 
diagnosing  leukemia.  "Knowledge"  from  an  expert  was 
extracted  from  transcripts  of  sessions  he  gave  during  a 
running  commentary  while  he  made  diagnoses  on  67  samples 
from  various  hospitals.  The  EMYCIN  system  (an  expert 
system)  was  translated  into  Edinburgh  University  DEC-10 
PROLOG  to  allow  the  rules  making  each  conclusion  to  be 
grouped  together  and  analyzed.  There  were  100  test  cases  of 
which  the  system  gave  satisfactory  answers  in  70  (70%) . 
Simple  or  hidden  errors  in  rules  were  corrected  and  the 
performance  improved  to  85/90  (94%) .  This  was  a  small 
system,  however*— handling  only  orthodox  cases.  A  high 
quality  system  must  handle  rare  cases  and  would  require  a 
more  comprehensive  representation  of  the  domain. 

Conclusion:  Expert  systems  offer  the  hope 

of  solving  problems  as  well  as  human  experts.  However,  the 
modest  success  in  this  system  does  not  mean  the  same  success 
will  result  in  a  large  system.  The  important  factor 
appeared  to  be  in  quality  control  at  all  stages  in  the 
development  of  the  expert  system  [Ref.  9]. 
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(2)  Automatic.  J£gg oh  -Fggqqnitidn*  An  expert 


system  incorporating  phonetic/phonological  and  lexical 
knowledge  was  developed  by  the  Speech  Group  in  the  Human- 
Computer  Interface  Research  Unit  at  Leicester  Polytechnic. 

The  spectrogram  was  chosen  as  the  initial  representation  of 
the  acoustic  wave-form  of  speech  in  which  communication  of 
this  knowledge  could  be  achieved.  The  expert  system  used 
PROLOG  rules  to  interpret  the  spectrogram  in  terms  of  both 
phonemes  and  words.  A  spectrogram  provides  a  visual 
representation  of  acoustic  waves  in  terms  of  the  intensity 
of  energy  presented  at  each  frequency  over  a  period  of  time 
The  program  was  encoded  by  an  expert  using  six  hundred 
spectrograms  produced  by  a  male  and  a  female  speaker.  It 
was  tested  on  twenty  spectrograms  of  continuous  speech  from 
four  different  speakers,  two  males  and  two  females.  The 
results  were  analyzed  in  terms  of  the  percentages  of 
phonemes  correctly  identified,  those  confused  with  other 
phonemes,  and  those  phonemes  which  were  simply  missed.  The 
overall  phoneme  recognition  rate  was  an  impressive  62%. 

Conclusion:  This  approach  can  achieve  more 

than  60%  correctly  recognized  phonemes  from  unknown 
utterances  on  a  speaker-independent  basis  and  is  therefore 
definitely  worth  pursuing  as  part  of  the  research  into 
speaker-independent  unrestricted  automatic  speech 
recognition  [Ref.  10], 


31 


(3)  Taxation.  The  development  of  ACCI,  a 
prototype  expert  system  in  the  area  of  taxation  known  as 
"apportionment  of  close  companies'  income,"  incorporates  tax 
legislation  explicitly,  together  with  the  control  structures 
relating  to  the  inter-dependency  of  statutes.  This  allows 
the  tax  inspector  to  obtain  advice  in  an  accurate,  cost- 
effective  and  natural  way.  One  of  the  primary  problems  in 
this  area  has  been  a  lack  of  experts  to  go  around  to  perform 
this  necessary  auditing  function.  The  main  benefits  of  this 
system  are  that  it  alleviates  the  problem  of  a  lack  of  human 
experts  in  this  area  of  taxation,  and  information  can  be 
obtained  much  more  efficiently  and  accurately.  ADVISOR  was 
the  expert  system  used,  which  meant  that  the  inference 
engine  of  the  system  was  already  provided.  The  knowledge 
for  this  system  was  obtained  from  four  sources:  Statutes? 
training  notes;  the  tax  inspector's  "field  manual";  and 
interviews  with  an  expert.  The  knowledge  base  was  built 
through  a  process  of  stepwise  refinement.  That  is,  a  piece 
of  legislation  is  first  considered  and  then  translated  into 
English-like  rules  which  are  subsequently  translated  into 
ADVISOR  rules.  Then,  the  rules  are  ordered  in  a  structure 
relevant  to  the  application  at  hand.  The  results  of  this 
system  were  very  impressive.  Consultations  that  were  taking 
the  tax  inspector  up  to  half  a  day  to  complete  were  reduced 
to  only  a  few  minutes  with  ACCI. 
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Conclusions:  ACCI  relieves  the  tax 
inspector  of  constantly  referring  to  a  manual  for  assistance 
and  provides  accurate  advice  quickly  and  easily.  ACCI 
therefore  demonstrates  the  effective  use  of  expert  systems 
technology  in  solving  problems  where  there  is  a  shortage  of 
human  experts  [Ref.  11]. 
b.  Military 

(1)  Budget  Analysis.  Department  of  Energy 
(DOE)  researchers  at  Oak  Ridge  National  Laboratory  in  Oak 
Ridge,  Tennessee  developed  one  of  the  first  expert  systems 
applications  for  the  government's  budget  process.  It  was 
created  for  the  Naval  Sea  Systems  Command  (NAVSEA)  as  a 
prototype  system  which  mimics  the  decision-making 
capabilities  of  budget  analysts.  The  expert  system  is 
called  BANS,  for  Budget  Analysts,  and  operates  on  IBM 
personal  computers.  The  NAVSEA  budgeting  branch  analysts 
are  responsible  for  management  and  control  of  all  aspects  of 
the  budget  for  a  group  of  28  activities.  Each  year,  they 
have  to  review  several  thousand  pages  of  budget  requests., 
identify  areas  with  increases  and  provide  justifications  for 
those  increases  within  a  two-week  period.  In  an  initial 
test  of  the  BANS  system,  a  budget  analysis  that  normally 
takes  two  to  three  hours  for  an  analyst  to  perform  took  BANS 
only  fifteen  minutes. 

Conclusion:  The  system  proved  to  work 

faster  than  the  analysts,  but  the  goal  is  for  greater 
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consistency  in  decision  making.  BANS  is  still  in  the 
testing  stage  and  will  be  used  to  analyze  the  same  problems 
analysts  handle.  The  results  will  be  compared  and  the 
system  is  expected  to  be  fully  operational  as  a  decision 
support  tool  in  the  fall  of  1987  [Ref.  12]. 

(2)  Steam  Propulsion  System.  One  of  the  first 
AI  efforts  in  the  area  of  training  was  the  Steamer  Project 
conducted  by  the  Naval  Personnel  Research  and  Development 
Center  (NPRDC)  in  San  Diego,  California.  The  Steamer 
Project  was  a  research  effort  concerned  with  implementing 
intelligent  computer  based  training.  A  steam  propulsion 
system  is  a  very  complex  physical  system,  and  the  propulsion 
spaces  take  up  about  one-third  of  the  space  in  most  Navy 
"ships.  Operating  the  plant  requires  a  supervisor  and  about 
sixteen  to  twenty-five  people  under  extremely  arduous 
circumstances.  It  takes  years  of  instruction  and  experience 
to  become  competent  in  this  area.  The  Steamer  Project  has 
been  used  as  a  training  aid  at  Great  Lakes  Training  Center 
and  Coronado  Island,  San  Diego,  California  with  very 
encouraging  results.  Preliminary  results  show  that 
personnel  respond  positively  to  the  program  and  can  learn  in 
a  much  shorter  period  of  time. 

Conclusion:  The  possibility  of  reducing 
the  training  cycle  and  maintenance  of  proficiency  through 
realistic  training  models  is  extremely  attractive  to  the 
military  services.  [Ref.  13] 
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5. 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 


Flexibility  of  Expression.  Expert  systems  can  embody 
rules-of-thumb  that  practitioners  tend  to  carry 
around  in  their  heads  but  never  write  down. 


Human-like  Processing.  Expert  systems  operate  terms 
and  concepts  that  an  individual  user  can  feel 
affinity  with  and  at  the  level  of  rules  and  facts, 
and  the  relationships  between  them,  rather  than  at 
the  level  of  program  steps. 

Ease  of  Expression.  Expert  systems  are  much  more 
easily  understood  by  practitioners  in  the  domain  who 
may  not  be  very  computer-literate. 

Uncertainty.  Uncertainty  and  contradictory 
evidence  are  handled  in  a  natural  way  in 
plausible-inference  systems.  So,  those  areas  where 
there  is  incomplete  knowledge  and  where  judgement  is 
needed,  which  is  so  prevalent  in  real  life,  can-,  be 
dealt  with  in  expert  systems. 

Checklist.  Expert  systems  do  not  forget,  unlike 
people,  and  can  reliably  pose  all  relevant  questions 
and  act  like  a  checklist. 


Refining  Expertise.  Most  experts  admit  to  having 
gaps  in  their  knowledge  and  expert  systems  can  help 
identify  where  these  lie. 

Communication  Medium.  Expert  systems  can  aid  in 
searching  and  processing  a  huge  collection  of  rules 
and  other  information.  In  addition,  it  can  form  the 
medium  for  sharing  experiences  between  two 
institutions.  (Ref.  7] 

Reduce  Costs.  Extracting  knowledge  from  experts  and 
translating  it  in  computer  form  can  greatly  reduce 
the  costs  of  knowledge  reproduction  and  exploitation. 

Performs  difficult  tasks  quickly  and  at  expert  levels 
of  performance. 

Employs  self-knowledge  to  reason  about  its  own 
inference  processes  and  can  therefore  provide 
explanations  or  justifications  for  conclusions  it 
reaches.  [Ref.  8] 
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b.  Disadvantages 

_ (1)  Expert  systems  cannot  learn  from  experience,  except 

in  trivial  ways. 

(2)  They  cannot  deal  with  analogies,  something  humans  do 
constantly. 

(3)  They  cannot  reason  intuitively,  which  is  what  makes 
human  experts  approach  the  level  of  artistry. 

(4)  Expert  systems  know  only  the  rules.  Experts  know  the 
rules  of  their  domain,  but  they  also  know  when  to 
break  them.  The  computer  cannot  exercise  the 
imagination  required  to  think  past  the  facts  and 
rules  in  its  possession.  [Ref.  6:pp.  41-44] 

(5)  It  is  difficult  to  ensure  knowledge  sufficiency  for 
the  planned  system.  The  real  power  of  expert  systems 
is  in  its  knowledge  base;  with  rapid  changes, 
fragmentation,  and  diversity  in  the  knowledge  of  the 
knowledge  base,  the  systems'  capabilities  can  be 
severely  hampered. 

(6)  It  is  difficult  to  find  an  expert  or  experts  who  have 
the  time  to  commit  to  such  a  formidable  effort  that 
is  required  to  build  a  knowledge  base.  [Ref.  4] 

(7)  Application  areas  have  not  yet  demonstrated 
high  returns. 


B.  THE  MODEL 

An  excellent  description  of  the  model  used  in  this 
thesis  can  be  found  in  LCDR  Joyce  Zellweger's  masters  thesis 
at  the  Naval  Postgraduate  School.  This  thesis  is  a 
follow-on  study  on  the  selection  of  successful  recruiters 
using  a  similar  model  as  adopted  in  Zellweger.  [Ref.  2:pp. 
76-91] 

Past  research  has  successfully  identified  attributes 
that  successful  recruiters  possess.  However,  nothing  has 
revealed  the  relative  importance  of  these  attributes  in 
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relation  to  each  other.  This  has  been  a  prime  concern  among 
researchers  and  is  the  basis  for  choosing  EXPERT8 7  as  the 
expert  system  in  which  to  conduct  this  analysis.  EXPERTS 7 
is  a  special  type  of  expert  system  based  upon  a  concept  its 
designer  has  labeled  Quasi-Artificial  Intelligence  (QAI) . 

QAI  differs  from  traditional  AI  in  that  it  is  not  intended 
for  the  breadth  of  problems  AI  systems  attempt  to  solve. 
Rather,  it  is  intended  for  the  large  class  of  moderately 
difficult  and  repetitive  decision  problems  which  so  often 
face  managers  and  decision  makers.  QAI  enables  coherent  and 
objective  decisions  to  be  made  when  no  known  criterion  or 
dependent  variable  is  available  for  the  development  of  an 
empirical  model.  It  allows  efficient  interaction  of  experts 
with  a  knowledge  base,  and  a  presentation  of  the  process  and 
results  in  a  form  which  can  be  understood  by  the  expert  or 
any  other  user  of  the  system. 

EXPERT87  is  not  a  simple  "weight  and  rate"  process, 
however.  It  is  not  decision  analysis,  nor  is  it  a 
cumbersome  and  time-consuming  AI  system  with  limited 
applicability.  Instead,  it  provides  a  format  for  gathering 
intuitive  knowledge  from  experts  in  minimal  time,  and  in  a 
manner  that  permits  verifiable  estimation  of  the  trustwor¬ 
thiness  of  the  expert  systems  that  emerge.  The  method  is 
based  on  mathematical  theory  that  allows  the  computer 
program  to  generate  hierarchically  ordered  profiles  of 
hypothetical  alternatives  (in  this  case,  recruiters). 
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Structuring  of  a  problem's  concepts  in  this  way  avoids 
cognitive  overload  of  experts  and  assures  a  more  beneficial 
utilization  of  attribute  information.  The  software 
generates  attribute  values  for  each  profile  or  alternative 
which  optimizes  the  likelihood  that  the  expert's  resulting 
model  correctly  represents  the  expert's  intuitive  knowledge. 

Several  important  principles  underlie  the  development 
of  EXPERT87 .  These  include  [Ref.  14:pp.  7-18]: 

1.  Intuition  is  a  component  of  thought  processes.  One 
of  the  basic  principles  of  decision  making  is  that 
people  have  sound,  intuitive  bases  for  acting  on 
given  problems,  even  though  they  seldom  objectively 
express  their  knowledge.  The  intuitive  knowledge 
they  possess  is  made  up  of  their  observations  about 
specific  attributes  of  a  problem,  which  they  seldom 
explicate  when  they  make  subjective  judgments  or 
evaluations. 

2.  Cognitive  abilities  are  limited.  In  a  well-known 
research  article  of  the  1950 's,  "The  Magic  Number 
Seven,  Plus  or  Minus  Two,”  psychologist  G. A.  Miller 
wrote  that  people  cannot  effectively  deal  with  more 
than  seven  concepts  at  a  time.  He  demonstrated  the 
validity  of  this  principle,  to  within  one  or  two 
categories,  across  a  wide  spectrum  of  human 
perceptual  and  cognitive  activities.  For  this 
reason,  EXPERT87  limits  problems  to  seven  concepts 

_ with  seven  attributes  per  concept.  Overloading 

human  information  processing  ability  is  counter¬ 
productive. 

3.  Cognitions  are  not  easily  communicated.  People  cannot 
clearly  communicate  their  thought  processes.  They  do 

not  know  what  information  is  routinely  ignored  or  _ 

discounted  in  their  thinking,  nor  do  they  know  how 
much  importance  they  attach  to  each  concept  when 
making  decisions.  When  asked  to  assign  weights  to  the 
factors  which  influence  their  decisions,  they  are 
often  hesitant  and  sometimes  unable  to  do  so. 

4.  There  is  a  mathematics  of  intuitive  processes.  The 
system  is  designed  to  overcome  the  difficulties 
described  above  and  attempts  to  capture  experts' 
intuitive  knowledge  without  forcing  them  to  think  like 
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mathematicians.  The  interactive  process  between 
expert  and  computer  generates  the  functionality 
between  attributes  of  alternatives  and  the 
overall  merit  of  the  alternatives.  The  functionality 
of  the  system  makes  it  unique,  an  expert  system  which 
is  not  ''rule-based”  in  the  usual  sense,  but  is 
function-based  in  the  sense  of  being  able  to  express 
in  simple  algebraic  form  the  fundamental  nature  of  the 
expert's  intuitive  processes.  The  system  responds  to 
each  new  decision  problem  using  a  functional  model  of 
expert  intuition  which  accurately  reflects  only  the 
consistent  and  reliable  components  of  these 
intuitions. 

5.  Hierarchies  express  relations  between  concepts  and 
attributes.  Hierarchies,  in  EXPERT37,  are  tree 
structures  carefully  designed  to  define  a  problem  in  a 
comprehensive,  meaningful,  and  well-organized  way. 
Solutions  are  represented  as  alternatives,  and  the 
expert's  job  is  to  evaluate  the  alternatives.  When 
the  program  evaluates  an  alternative,  it  calibrates 
the  value  of  that  alternative  in  terms  of  the  expert's 
own  set  of  primary  defining  concepts 

and  attributes.  According  to  L.  L.  Whyte,  hierarchies 
are  the  most  powerful  method  of  classification  used  by 
the  human  brain  in  ordering  experience,  observations, 
entities  and  information. 

6.  Hypothetical  constructs  can  be  mapped  into  intervening 
variables.  The  difference  between  concepts  and 
attributes  is  essential  to  understanding  EXPERTS 7 . 
Attributes  are  specific,  people  agree  on  their 
meaning,  and  people  are  relatively  reliable  and 
consistent  when  rating  attributes.  Concepts  are 
general  and  people  tend  to  impose  their  own  unique 
meaning  on  the  concepts  they  use.  As  a  result, 
people  usually  argue  about  choices  among  alternatives 
because  they  fail  to  understand  that  while  other 
persons  may  be  using  the  same  word  to  refer  to  a 
concept,  the  concept  is  not  identical  to  their  own. 
Psychologists  usually  use  the  terms  hypothetical 
construct  and  intervening  variable  to  clarify  the 
distinction  between  unquantified  ideas  and 
operationally  defined  measures.  Figure  2  depicts  an 
individual's  construct  (labeled  "V")  as  a  somewhat 
vague  and  incompletely  specified  set  of  ideas.  The 
figure  illustrates  the  explication  of  the  construct, 
first  by  defining  it  in  terms  of  a  set  of  measurable 
attributes,  and  then  in  the  generation  of  an  expert 
system.  In  EXPERT87 ,  expert  systems  are  functional 
definitions  of  constructs  and  contain  processes  for 
mapping  information  from  measurable  attributes  into 
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Figure  2.  Transforming  Concepts/Constructs  into 
Variables  [Ref.  14:p.  81] 


measures,  which  are  labelled  intervening  variables. 
Then,  it  can  be  said  that  the  measure,  V,  is  an 
operational  definition  of  the  construct.  Figure  3 
illustrates  the  substitution  of  hypothetical 
constructs  as  concepts  of  a  hierarchy.  The  problem  is 
that  the  linkage  between  attributes  and  constructs  is 
missing.  EXPERT87  constructs  an  expert  system  to 
provide  this  linkage  after  interacting  with  an 
expert  and  utilizing  its  newly-discovered  knowledge 
about  the  way  the  expert  responds  to  problems.  For 
completeness,  each  concept  is  linked  with  its  set  of 
subordinate  attributes.  The  expert  system  must  also 
map  the  derived  values  of  the  concepts  into  an  over¬ 
all  assessment  of  every  alternative  of  interest.  This 
representation  is  displayed  in  Figure  4.  The  bottom 

_ level  attributes  of  communication*  skills  are  assumed 

to  be  easily  observed  and  rated.  The  remaining  levels 
represent  indirect  measurement  processes,  governed  by 
expert  systems. 

.  Effective  decision  aides  develop  understanding  and 
confidence.  Cognitive  psychologists  developed  the 
underlying  principles  of  EXPERT87  over  a  lengthy 
period  of  time.  These  principles  enable  the  user  to 
implement  the  formal  representation  of  decision 
processes  on  a  microcomputer,  to  make  explicit  the 
intuitive  knowledge  and  expertise  of  decision-makers, 
and  to  structure  decision  problems  in  hierarchical 
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Attributes 


Figure  3.  Defining  Problems  in  Terms  of  Hierarchies 
[Ref.  14 :p.  82] 


form.  The  user  is  then  able  to  define  terms  and 
concepts  and  to  clarify  his/her  thinking; 
to  develop  a  clearer  understanding  of  the  consistent 
and  reliable  components  of  the  expert's  intuitive 
reactions  to  sets  of  information.  Furthermore,  once 
an  expert  system  is  constructed,  it  is  there  to  guide 
the  evaluation  of  every  new  alternative  that  comes 
along  in  the  future,  to  evaluate  thinking  about  an 
alternative,  and  to  help  choose  among  alternatives. 

The  analytic  routines  EXPERT87  utilizes  identifies 
consistencies  in  the  decision  maker's  subjective  judgments, 
and  expresses  these  consistencies  in  terms  of  the  expert's 
concepts  and  values.  The  system  then  characterizes  the 
user's  intuitive  processes  in  easy  to  understand  algebraic 
and/or  numeric  form. 

There  are  technical  limitations  to  EXPERT87,  and  as  in 
all  commercially  available  expert  systems  software,  it  is 
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_ Evaluation  of  Alternatives 

using  EXPERTS 7 


Figure  4.  Schematic  of  Expert  System  Development  at 
Two  Levels  of  a  Hierarchy  [Ref.  14:p.  16] 


proprietary.  Therefore,  detailed  information  about  its 
algorithms  and  operations  is  limited. 

In  summary,  EXPERT87  was  chosen  for  this  thesis 
because: 

-  the  software  is  easy  to  use  and  understand 

-  an  expert  system  is  developed  easily  and  quickly, 
feedback  is  immediate,  and  results  are  easy  to  interpret 

-  the  software  can  handle  a  wide  variety  of  decision¬ 
making  problems,  and 
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-  the  program's  cognitive  engine  is  deductive  rather 
than  inductive,  which  stimulates  human  thinking  and 
reasoning  more  accurately  than  the  more  traditional 
expert  systems. 

Figure  5  depicts  the  specific  hierarchy  developed  to 
model  the  profile  of  a  successful  recruiter.  The  goal  of 
the  model  is  to  identify  and  weight  characteristics  of  the 
successful  recruiter,  which  is  the  node  at  the  top  of  the 
hierarchy  labelled  "Profile  of  the  Successful  Recruiter." 

Based  on  the  literature  review  as  well  as  discussions 
with  field  recruiters,  characteristics  believed  to  be 
related  to  recruiter  success  have  been  identified  (Table  2 
in  Chapter  I).  These  characteristics  are  related,,  and  can 
be  organized  into  logical  categories.  These  are 
Communications  Skills,  Demographic  Characteristics,  Military 
Background,  Personality  Characteristics,  Behavior 
Characteristics,  and  Specific  Experience.  These  dimensions 
become  the  largest  branches,  or  nodes,  of  the  hierarchy. 

The  characteristics,  or  attributes,  within  each  dimension 
become  the  smaller  nodes  of  the  hierarchy.  A  description  of 
the  attributes  within  each  dimension  are  characterized 
below: 

1.  Communication  Skills 

-  Public  Speaking  Skills 

The  ability  of  a  recruiter  to  stand  before  a  group 
of  people  and  convey  information  so  as  to  motivate 
an  audience  is  thought  to  be  an  attribute  a  success 
recruiter  possesses. 
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Figure  5.  Profile  of  the  Successful  Recruiter:  The  Model 


-  Writing  Skills 

Although  a  recruiter's  job  involves  very  little 
writing,  writing  is  such  a  large  part  of 
communicating,  that  it  was  included  in  the  model. 

-  Listening  Skills 

Many  of  the  recruiters  who  tested  this  model 
believe  that  listening  skills  are  the  most 
important  aspect  of  a  recruiter's  communication. 

By  asking  open-ended  questions  and  carefully 
listening  to  an  applicant,  the  successful  recruiter 
can  provide  information  targeted  specifically  at 
the  needs  and  desires  identified  by  the  individual. 

-  Informing 

The  successful  recruiter  has  the  ability  to  recall 
information  necessary  to  effectively  inform  the 
applicant  on  all  aspects  of  military  life. 

-  Persuading 

The  successful  recruiter  must  be  able  to  close  the 
sale. 

Demographic  Characteristics 

-  Age 

Experience  is  thought  to  come  with  age.  However, 
if  the  recruiter  is  too  old,  he/she  may  not  be 
able  to  relate  to  a  young  applicant;  if  too 
young,  the  recruiter  does  not  have  enough 
experience  to  help  an  applicant. 

-  Family  Support 

An  aspect  of  recruiting  that  affects  the 
probability  that  a  recruiter  will  be  successful 
is  the  issue  of  family  support,  particularly  of 
the  spouse.  Recruiting  duty  often  means  living 
in  areas  away  from  a  military  community  and 
services  the  family  depends  upon.  Living  away 
from  military  commissaries,  exchanges,  and 
medical  facilities  can  create  or  increase 
financial  hardship  and  stress  for  families. 
Recruiting  also  involves  long  hours,  weekend 
work,  and  travel  away  from  home. 
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AFQT 


Most  of  the  literature  suggests  that  intelligence 
is  directly  related  to  recruiter  success — the 
higher  the  AFQT  scores  the  recruiter  has,  the 
better. 

-  College  Experience 

Education  and  ASVAB  scores  are  often  used  as 
readily  available  measures  of  intelligence. 

3.  Military  Background 

-  Pay-grade 

The  recruiters  who  tested  this  model  all  felt  that 
the  most  successful  recruiters  are  E-6's.  E-5's 

and  E-7's  are  next,  and  E-8's  and  E-9's  last. 

E-4's  and  below  do  not  possess  the  necessary 
experience  to  be  successful  and  E-8's  and  E-9's 
tend  to  intimidate  applicants. 

Years  of  Service  (Active) 

With  respect  to  age  and  years  of  service,  a 
recruiter  must  have  experience  in  the  service  and 
be  old  enough  to  have  some  credibility. 

-  Years  of  Service  (Reserve) 

For  reserve  recruiters,  some  experience  in  a 
reserve  unit  is  necessary  for  the  recruiter  to  sell 
the  candidate  on  reserve  life. 

4.  Personality  Characteristics 

-  Self-Image 

The  successful  recruiter  has  a  positive  self-image 
and  outstanding  military  bearing.  One  recruiter 
noted  that  a  sloppy  looking  recruiter  can 
immediately  turn  off  an  applicant. 

Integrity 

This  attribute  showed  up  most  often  as  the  most 
important  characteristic  within  this  dimension. 

The  recruiter  who  lacks  this  attribute  is  also  the 
one  with  the  most  fraudulent  enlistments  and  is 
almost  immediately  weeded  out  of  the  recruiting 
game . 
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Extroverted 


The  successful  recruiter  is  more  concerned  for 
others  than  himself/herself . 

-  Sense  of  Humor 

This  may  help  a  recruiter  enjoy  the  job,  and  may 
help  keep  him/her  on  an  even  keel  in  a  very 
demanding  job. 

-  People-Oriented 

The  successful  recruiter  enjoys  working  with 
people. 

5.  Behavior  Characteristics 

-  Self-starter 

A  recruiter's  job  entails  doing  just  about 
everything  on  his/her  own.  The  recruiter  must  be 
able  to  motivate  himself/herself  to  get  the  job 
done. 

Commitment 

To  be  successful,  the  recruiter  must  like  his/her 
job  and  be  committed  to  it. 

-  Flexibility 

A  successful  recruiter  must  be  able  to  adapt  to 
his/her  environment  and  change  plans  on  a  moment's 
notice. 

Attention  to  Detail 

To  be  successful,  the  recruiter  should  be  able  to 
plan  activities  over  various  time  periods.  He/she 
must  also  be  organized  so  as  not  to  forget  a  single 
detail . 

Decisiveness 

The  successful  recruiter  must  be  able  to  make  a 
decision  on  his/her  own. 
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6.  Specific  Experience 

-  Sales  Experience 

Civilian  sales  experience  may  be  a  substitute  for 
recruiting  experience,  since  recruiters  are  often 
described  as  salespeople. 

-  Public  Speaking  Experience 

A  person  with  public  speaking  experience  has  an 
advantage  over  other  recruiters. 

-  Counselling  Experience 

A  recruiter  with  prior  counselling  experience  also 
has  an  advantage  over  other  recruiters. 

For  each  of  the  six  dimensions  described,  EXPERT87  will 
generate  a  number  of  hypothetical  profiles  which  each  expert 
will  evaluate.  The  software  takes  tha  expert  through 
evaluations  of  attributes  within  each  dimension  and 
evaluations  of  the  relative  importance  of  dimensions.  For 
example,  the  software  will  generate  a  specially  constructed 
set  of  attribute  values  for  each  attribute  which  defines  the 
dimension.  The  larger  the  number  of  attributes  within  the 
dimension,  the  more  profiles  the  systam  will  generate  for 
expert  assessment.  This  is  necessary  to  provide  a  sample 
size  sufficient  to  allow  for  a  valid  model.  Each  profile  is 
presented  in  graphic  form  for  the  expert  to  examine,  reflect 
on,  and  assess,  as  depicted  in  Figure  6.  For  each 
dimension,  experts  use  their  own  knowledge,  experience,  and 
intuition  to  evaluate  individuals  having  profiles  of 
attributes  for  that  dimension.  The  assessment  is  based  on 
the  scale  shown  at  the  bottom  of  the  graph.  The  expert 
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enters  a  score  from  00  to  99  depending  on  his/her  overall 
evaluation  of  that  individual.  This  procedure  is  then 
repeated  for  each  dimension  in  the  model. 

Once  the  last  profile  has  been  evaluated,  the  software 
completes  its  mathematical  routines  and  stores  functional 
relationships  between  attributes  and  dimensions.  Now  that 
the  expert  system  is  in  place,  it  can  evaluate  real 
alternatives  based  on  each  expert's  expertise.  An 
additional  profile  is  displayed  and  evaluated  based  on  the 
expert  system  just  created.  After  the  expert  enters  his/her 
assessment,  the  system  displays  its  predicted  value  of  that 
expert's  assessment.  With  reasonable  care,  the  expert's 
response  should  be  accurate  to  within  five  or  six  percent  of 
the  system's  findings.  For  example,  a  small  mean  squared 
error  of  5.0  indicates  that  the  expert  system  can  predict 
the  expert's  assessment  to  within  plus  or  minus  five  points 
or  67  percent  of  the  time  and  plus  or  minus  ten  points  99% 
of  the  time.  The  mean  squared  error  also  identifies 
instances  when  an  expert  might  have  made  a  gross  error  in 
entering  an  evaluation. 

One  of  the  most  important  evaluation  tools  contained  in 
EXPERT87  is  the  Fidelity  Index.  This  index  indicates  how 
successful  the  program  was  in  developing  an  expert  system 
that  correctly  models  the  expert's  own  ii. tuitions.  If 
Fidelity  is  less  than  80  percent,  there  is  a  strong 
indication  that  the  expert's  evaluations  were  inconsistent, 
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which  means  that  the  intuitive  or  cognitive  processes 
underlying  the  expert's  assessments  were  not  used  in  a 
consistent  way. 

Relative  weights  are  also  calculated  for  each  expert, 
indicating  the  relative  importance  of  each  attribute  or 
dimension.  The  expert's  judgments  are  said  to  be  a 
monotonic  function  of  the  attribute  values.  That  is,  as  the 
value  of  an  attribute  increases,  the  value  of  the  concept  to 
which  it  belongs  either  increases  or  decreases.  The 
software  also  determines  (for  each  expert)  the  shape  of  the 
function  of  each  attribute,  whether  it  is  positive  or 
negative,  monotonic  or  non-monotonic,  linear,  convex,  or 
concave. 

This  information  provides  the  expert  with  a  better 
understanding  of  his/her  intuitive  processes  and  personal 
values.  The  sign  of  the  non-linear  component  is  actually 
the  second  derivative  of  the  concept  under  evaluation  with 
respect  to  the  attribute.  Positive  signs  indicate  U-shaped 
functions,  and  negative  signs  indicate  functions  which  rise 
to  a  peak,  accelerating  at  a  decreasing  pace,  and  then 
reversing.  So,  if  an  attribute  has  a  relatively  large 
linear  component,  this  means  that  the  value  of  the  concept 
increases  linearly  with  the  magnitude  of  the  attribute.  If 
the  relative  weights  also  contain  a  significant  negative 
non-linear  component,  this  implies  a  leveling  off,  or  a 
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reversal  of  this  trend  for  the  larger  magnitudes  of  the 
attribute.  [Ref.  14:pp.  84-65] 

At  no  time  does  EXPERT87  ask  the  expert  to  indicate  how 
important  each  attribute  is.  This  information  is  generated 
by  the  program  based  on  the  expert's  evaluations  of  profiles 
of  individuals  with  specific  measured  quantities  of  each 
attribute.  The  fidelity  index  is  then  used  as  an  indicator 
of  how  accurate  EXPERT87  simulates  the  expert.  [Ref.  14 :p. 
85] 

C.  THE  EXPERTS 

An  "expert"  is  a  person  who,  because  of  training  and 
experience,  is  able  to  do  things  the  rest  of  us  cannot.  An 
expert  is  not  only  proficient  but  also  smooth  and  efficient 
in  the  actions  he/she  takes.  He/she  knows  a  great  many 
things  and  has  tricks  and  caveats  for  applying  knowledge  to 
problems  and  tasks.  An  expert  is  also  good  at  plowing 
through  irrelevant  information  in  order  to  get  at  basic 
issues  and  recognizing  problems  as  instances  of  types  which 
are  familiar.  [Ref.  15:p.  5] 

In  the  recruiter  selection  problem,  the  experts  are 
recruiters  who  are  either  doing  the  job  and  have  succeeded 
themselves,  or  recruiter  instructors  who  have  been 
successful  in  the  field.  Experts  1  through  6  are  AGR 
recruiters,  with  experts  1,  2,  4,  and  5  being  field 
recruiters  from  Indiana  battalions,  and  experts  3  and  6  are 
recruiter  instructors  at  the  recruiter  school  in 
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Indianapolis.  Experts  7  through  16  are  active  duty 
recruiters,  all  of  which  are  instructors  at  the  recruiter 
school . 

The  next  chapter  will  analyze  the  similarities  and 
differences  of  each  of  the  expert  systems  created  by  the 
sixteen  experts.  In  addition,  a  composite  model  using  the 
mean  scores  of  the  sixteen  experts  will  be  analyzed  and 
contrasted  with  the  individual  models  as  well  as  with  a 
"patchwork"  model  (the  patchwork  model  represents  the  "best" 
of  the  experts  and  will  be  explained  in  more  detail  in  the 
next  chapter) .  Trends  and  relative  weights  among  dimensions 
and  attributes  will  also  be  analyzed  to  determine  if  a 
consistent,  clearly  identifiable  profile  of  a  successful 
recruiter  emerges.  Finally,  an  evaluation  of  hypothetical 
recruiter  applicants  will  be  examined  to  compare  the 
different  experts'  ratings  of  the  same  applicants. 
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III.  ANALYSIS  AND  RESULTS 


A.  THE  EXPERTS 

The  expert  system  developed  for  each  of  the  sixteen 
recruiter  experts  will  be  evaluated  and  compared  in  terms  of 
Fidelity,  Standards,  and  Discrimination  (as  explained  in 
detail  in  the  previous  chapter) .  These  indices  typically 
range  in  value  from  0  to  100  and  can  be  used  to  interpret 
the  worth  of  the  expert  system.  (It  is  important  to  note 
that  the  version  of  EXPERT87  used  in  this  thesis  did  not 
incorporate  a  normalization  process  for  these  indices  and 
the  values  sometimes  exceed  100  or  go  below  0.  (Version  4.2 
will  correct  this  problem.) 

As  previously  described,  the  fidelity  index  measures  how 
well  the  expert  system  correctly  reproduces  the  intuitive 
—judgments  of  the  expert,  the  standards  index  measures  the 
extent  to  which  the  experts  maintain  high  standards  on  their 
assessments  of  hypothetical  profiles  of  recruiters,  and  the 
discrimination  index  measures  the  expert's  ability  to  make 
fine  distinctions  among  hypothetical  profiles  of  recruiters. 
An  evaluation  of  the  perfect  index  for  each  of  the  three 
concepts  must  be  left  to  the  user  of  the  system.  For 
example,  if  the  user  wants  an  expert  who's  standards  are 
extremely  high,  he/she  would  choose  an  expert  with  a 
standards  index  above  80.  Similarly,  the  user  would  choose 
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an  expert  with  a  discrimination  index  above  80  if  he/she 
wants  someone  who  is  highly  discerning.  A  fidelity  index 
closer  to  100  is  best  in  any  situation,  with  80  being 
marginally  acceptable  and  40  being  very  inconsistent. 

Appendix  B  displays,  for  each  of  the  sixteen  expert 
systems,  the  three  indices,  the  mean  squared  error,  and  the 
explained  variance  in  each  of  the  six  dimensions  and  the 
overall  model. 

For  the  sixteen  expert  systems,  the  overall  model 
Fidelity  Index  was  above  90  in  eleven  cases  and  above  83  in 
the  remaining  five  cases.  For  the  remaining  dimensions 
(Communication  Skills,  Demographic  Characteristics,  Military 
Background,  Personality,  Behavior,  and  Specific  Experience) , 
the  Fidelity  Index  remained  above  80  in  all  but  seven  cases, 
with  the  lowest  being  69.1  on  Expert  14 's  assessment  of  the 
Military  Background  dimension.  Other  results  with  Fidelity 
Indices  below  80  were:  Expert  1,  77.8  on  the  Specific 
Experience  dimension;  Expert  4,  75.4  on  the  Demographic 
Characteristics  dimension;  Expert  8,  78.3  on  the  Personality 
Characteristics  dimension;  Expert  9,  78.1  on  the  Demographic 
Characteristics  dimension  and  75.4  on  the  Personality 
Characteristics  dimension;  and  Expert  14,  79.6  on  the 
Behavior  Characteristics  dimension.  Many  of  these 
dimensions  were  either  unimportant  in  the  eyes  of  the 
experts  or  least  important  in  relation  to  the  other 
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dimensions  and  therefore  perhaps  the  experts  did  not 
evaluate  them  as  carefully  as  the  others. 

For  the  overall  model,  the  expert's  Standards  Indices 
ranged  form  17.8  to  99.3.  A  regular  active  duty  recruiter 
(RA)  Instructor/Guidance  Counselor  was  the  most  lenient,  and 
an  AGR  Recruiter  Instructor  had  the  highest  Standards  Index. 
The  Standards  Indices  for  the  rest  of  the  dimensions  varied 
for  all  experts,  but  those  experts  whose  Standards  Indices 
were  high  for  the  overall  model  tended  to  have  higher 
indices  than  the  other  experts  for  the  individual  dimensions 
as  well. 

The  experts'  Discrimination  Indices  ranged  from  18.1  to 
103.8.  Expert  12,  the  RA  Instructor/Guidance  counselor  had 
the  highest  Discrimination  Index.  Expert  1,  an  AGR  field 
recruiter  had  the  lowest. 

B.  DIMENSIONS 

Table  3  (AGR  recruiters)  and  Table  4  (RA  recruiters) ~ 
present  the  relative  weights  assigned  to  the  model's  six 
dimensions  by  each  of  the  sixteen  experts.  The  weights  in 
each  column  will  sum  to  approximately  one  and  can  be 
interpreted  as  the  relative  importance  of  one  attribute  in 
relation  to  the  others.  For  example,  Expert  l's  weight  for 
the  Demographic  Characteristics  dimension  is  .321,  which  is 
approximately  three  times  as  important  as  the  Behavior 
Characteristics  dimension  which  has  a  relative  weight  of 
.05.  Communication  Skills  (.305)  is  approximately  three 


56 


as 

00 

as 

< 

*r 

r* 

as 

00 

« 

o 

o 

rH 

rH 

E 

• 

• 

• 

• 

• 

CM 

cn 

as 

as 

o 

r- 

*-4 

in 

U3 

'O' 

CN 

rH 

o 

o 

r-i 

t 

• 

• 

• 

• 

CN 

r* 

CN 

rH 

00 

so 

SO 

CN 

ro 

m 

CN 

rH 

rH 

CN 

rH 

rH 

• 

• 

• 

• 

• 

rH 

<J\ 

as 

00 

00 

00 

00 

P* 

o 

"T 

ro 

o 

o 

O 

CN 

rH 

• 

• 

• 

• 

• 

r~ 

CN 

m 

in 

as 

o 

O 

ro 

CN 

o 

rH 

CN 

CN 

• 

• 

• 

• 

• 

z 

O 

w 

w 

00 

r* 

CN 

o 

00 

z 

00 

ON 

00 

rH 

u 

CN 

*T 

o 

o 

o 

rH 

E 

rH 

• 

• 

• 

• 

• 

M 

a 

>, 

T3 

b. 

C 

O 

o 

rH 

in 

in 

o 

M 

o 

CN 

GO 

m 

o 

H 

1 

rH 

ro 

rH 

rH 

rH 

ro 

J 

cn 

a 

rH 

• 

• 

• 

• 

• 

a 

eh 

> 

< 

•H 

E-i 

<3 

4-> 

M 

0 

SO 

in 

O 

w 

< 

*T 

m 

o 

SO 

rH 

2 

o 

in 

o 

rH 

o 

CN 

rH 

• 

• 

• 

• 

• 

tJ 

> 

M 

H 

rH 

CN 

ro 

SO 

CN 

< 

<-H 

r* 

m 

as 

in 

J 

CT» 

fH 

rH 

rH 

CN 

rH 

U 

• 

• 

• 

• 

• 

a 

r*" 

CO 

m 

co 

ro 

m 

o 

00 

o 

o 

CN 

rH 

• 

• 

• 

• 

• 

ro 

00 

o 

rH 

00 

<Jl 

co 

m 

r* 

r- 

CN 

o 

rH 

CN 

rH 

■  * 

• 

• 

• 

• 

03 

rH 

rH 

• 

4J  c 

•H 

u  0 

0 

a  *h 

C/3 

• 

f0 

« 

0 

a  m 

o 

CQ 

C 

x  c 

• 

•  CO 

0 

> 

a  <u 

P 

0  U 

• 

w 

<0 

£ 

c 

£  fd 

rH 

u 

£3 

-H 

0 

<u  x: 

•H 

0 

Q 

Q 

u 

Q  U 

2 

a, 

CQ 

58 


Spec.  Exp.  .061  .118  .116  .021  .020  .175  .075  .159  .071  .071  .056 


times  as  important  as  Specific  Experience  (.065)  in  a 
successful  recruiter.  A  more  detailed  data  display  for  each 
expert  is  contained  in  Appendix  C. 

Table  5  (AGR  recruiters)  and  Table  6  (RA  recruiters) 
display  the  expert  systems’  most  important,  second  most 
important,  and  least  important  dimensions  along  with  their 
relative  weights.  By  order  of  importance.  Communication 
Skills,  Demographic  Characteristics,  and  Personality 
Characteristics  were  the  most  important  dimensions  to  the 
AGR  Recruiters.  Military  Background  and  Behavior 
Characteristics  were  relatively  less  important,  and  Specific 
Experience  was  judged  least  important  of  the  six  dimensions. 
The  active  recruiters  judged  Communication  Skills, 
Personality  Characteristics,  and  Behavior  Characteristics  as 
the  most  important  dimensions.  Military  Background, 

Specific  Experience,  and  Demographic  Characteristics  were 
all  significantly  less  important.  Hence,  for  dimensions, 
the  main  difference  between  RA  and  AGR  recruiters  was  in  the 
Demographic  dimension.  The  AGR  recruiters  felt  it  was 
important  and  the  RA  recruiters  felt  it  was  significantly 
less  important. 

C.  ATTRIBUTES 

The  importance  of  the  attributes  within  each  of  the  six 
dimensions  will  be  discussed  in  this  section.  Again, 
further  detail  for  all  attributes  and  dimensions  is 
contained  in  Appendix  C. 
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TABLE  ( 


HOST/LEAST  IMPORTANT  DIMENSIONS 


Regular  Arm' 


Expert 

7 

8 

9 

10 

11 

12 

13 

Most 

Important 

Common. 

.  2  S  3 

Commun , 
.437 

Person. 

.  296 

Commun . 

.  546 

Behav. 

.  305 

Comanjn. 

.448 

Commun. 

.297 

Second  Moat 
Important 

Person . 
.250 

Person. 

.257 

Demogr . 
.172 

Behav . 
.214 

Commun. 

.300 

S.Exp. 

.175 

Person. 

.275 

Least 

Important 

S . Exp . 
.061 

Demogr  . 
.048 

Common . 

•  Ill 

S.Exp. 

.021 

S.Exp. 

.020 

Person. 

.080 

Demogr. 

.075 

Expert 

14 

15 

16 

Mean 

Most 

I mpor  tant 

Commun . 

.  381 

Commun . 
.242 

Commun. 

.  402 

Commun. 

.434 

Second  Most 

1 mpor  tant 

Behav . 

.  208 

person . 
.221 

Demogr . 
.272 

Person. 

.194 

Least 

I mpor  tant 

Person . 
.074 

S . Exp. 
.071 

Behav . 
.071 

Demogr . 
.0498 
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The  attributes  within  the  Communication  Skills 
dimension  are  Public  Speaking  Skills,  Writing  Skills, 
Listening  Skills,  Informing,  and  Persuading.  Table  7 
displays  the  experts'  judgments  about  the  attributes  of  the 
Communications  Skills  dimension.  Three  of  the  six  AGR 
Recruiters  judged  Persuading  most  important,  while  two 
thought  Listening  was  most  important,  and  one  felt  Informing 
was  the  most  important  communication  skill.  Five  of  the  six 
AGR  Recruiters  felt  writing  skills  was  the  least  important 
attribute  in  this  dimension,  and  one  said  public  speaking 
skills  was  least  important. 

Similarly,  the  Active  Recruiters  judged  Persuading 
as  most  important  in  seven  of  the  ten  cases.  Two  felt 
listening  was  most  important  and  one  said  informing  was  the 
most  important  attribute  within  the  Communication  Skills 
dimension.  The  Active  Recruiters  also  felt  that  Writing 
Skills  were  least  important  in  half  the  cases.  Public 
Speaking  Skills  least  important  in  three  cases,  Persuading 
was  least  important  in  one  case,  and  Informing  was  judged 
least  important  in  one  case.  The  latter  two  were  totally 
contrary  to  the  consensus  of  opinions  among  the  majority  of 
recruiters  interviewed. 

2 .  Personality  Characteristics 

The  Personality  Characteristics  dimension  includes 
Self-Image,  Integrity,  Extroverted,  Sense  of  Humor,  and 
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TABLE  7 


COMMUNICATIONS  SKILLS  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reser  ves 


Exp* 1 1 

1 

2 
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4 

5 
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Mean 

M03t 

Important 

Inform. 

.  291 

Listen  . 

.  3  5  3 

Persuad . 

.  357 

Persuad . 

.  467 

Listen. 

.444 

Persuad . 

.  397 

Persuad . 
.31) 

Least 

Important 

Wr Itlng 
.063 

Writing 

.028 

Wr i t Ing 
.041 

Writing 

.088 

Writing 

.051 

P  .Spk ing 
.016 

Writing 

.041 

Requ lar  Army 


Expert 

7 

8 

9 

10 

11 

am 

13 
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I mpor  tant 

Persuad . 
.333 

Inform. 

.319 

Persuad . 

.  466 

Persuad . 
.370 

Listen. 

.318 

BBS! 

Persuad . 
.440 

Least 

I mpor  taut 

Wr i t i ng 
.  106 

Persuad . 
.08  3 

P.Spk int 
.056 

Writing 

.061 

P .  Spk  Inc 
.092 

Inform. 

.079 

Wr  1 1  ing 
.033 

Expert 

14 

15 

16 

Mean 

Most 

I mpor  tant 

Listen. 

.  332 

Persuad. 

.  311 

Persuad . 

.  346 

Persuad . 
.364 

L*ast 

I mpor  tant 

P  .  Spk 1 ng 
.085 

Writing 
.  068 

Writing 

.024 

Writing 

.043 
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People-Oriented.  As  shown  in  Table  8,  both  the  AGR  and 
Active  Recruiters  consistently  identified  Integrity  as  the 
most  important  attribute  within  the  Personality 
Characteristics  dimension.  In  fact.  Expert  18  judged 
Integrity  to  be  eight  times  more  important  than  Sense  of 
Humor,  which  was  the  least  important  attribute.  Sense  of 
Humor  and  People-Oriented  were  consistently  judged  as  the 
least  important  attribute  within  this  dimension.  Most 
recruiters  felt  that  a  sense  of  humor  is  nice  to  have,  but 
not  important  to  success.  The  author  felt  it  would  be 
important  to  deal  with  the  high  stress  factor  recruiters 
contend  with  every  day. 

3.  Behavior  Characteristics 

The  attributes  within  this  dimension  are  Self- 
Starter,  Commitment,  Flexibility,  Attention  to  Detail,  and 
Decisiveness.  Table  9  reveals  the  judgments  within  the 
Behavior  Characteristics  dimension.  AGR  and  Active 
Recruiters  again  agree  on  their  judgments  of  Self-starter 
and  Commitment  as  the  number  one  attribute  within  this 
dimension.  The  least  important  attribute  has  more 
variation,  but  Decisiveness  and  Flexibility  appear  most 
often.  The  wide  variations  in  these  results  is  probably  due 
to  the  expert's  opinions  that  all  of  these  attributes  have 
fairly  uniform  importance. 
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TABLE  8 


PERSONALITY  CHAR ACTER I STICS  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reserves 


Expert 

1 

2 

3 

4 

5 

6 

Mean 

Most 

Image 

mm 

I nt eg . 

I nteg . 

mm 

mm 

Important 

.  3n 

mm 

.4  49 

.432 

HQmH 

Least 

People 

Extr  ov . 

Humor 

Extrov. 

Humor 

Extrov . 

Humor 

Important 

.0S6 

.029 

.054 

.061 

.020 

.094 

.074 

Requ  lar  Arm' 


Expert 

7 

8 

9 

10 

11 

12 

13 

Most 

Important 

Integ. 

.417 

Integ . 
.352 

Extrov . 
.339 

I nteg . 
.537 

BB 
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.812 
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.505 
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.045 
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.066 
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.032 
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.093 
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Expert 

15 
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m 

Integ. 

.  583 

m 

Extrov. 

.507 

Least 

I mpor  tan t 

People 

.062 

Humor 

.039 

Humor 

.064 
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TABLE  9 


BEHAVIOR  CHARACTERISTICS  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reserves 


Expert 

1 

2 

3 

4 

5 

6 

Mean 

Most 

Important 

Detail 

.433 

Commit . 

.  402 

Starter 

.328 

Starter 
.  307 

Starter 

.251 

Flex. 

.  289 

Star  ter 
.  301 

Least 

Important 

Flex. 

.051 

Decis . 
.101 

Decis. 

.041 

Decis . 
.115 

Deta  1 1 
.093 

Decis . 
.076 

Regular  Arm1 


Expert 

7 

3 

9 

10 

11 

12 

13 

Most 

Important 

Starter 
.  279 

Flex. 

.  380 

Starter 

.345 

Deta  i  1 
.  569 

Starter 

.353 

Starter 

.541 

Least 

Important 

Decis . 
.119 

_ 

Decis. 

.060 

Flex. 

.031 

Starter 

.038 

Deta  i  1 
.105 

Decis . 
.035 

Deta  1 1 
.105 

Expert 

14 

15 

16 

Mean 

Most 

Important 

Flex. 

.332 

Commit . 
.277 

Detail 

.458 

Starter 

.312 

Least 

Important 

Flex. 

.098 

Commit . 
.091 

ESDI 

mSSm 

Decis . 
.087 
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4.  Military  Background 

Military  Background  attributes  include  Paygrade, 
Years  of  Service  (active)  and  Years  of  Service  (reserve) . 
Among  these  attributes,  Paygrade  was  most  important  to  AGR 
Recruiters  and  Years  of  Service  (Active)  was  least 
important.  Active  Recruiters  judged  Paygrade  and  Years  of 
Service  (Active)  to  be  most  important  and  Years  of  Service 
(Reserve)  least  important.  These  results  are  not 
surprising.  All  three  attributes  measure  experience  in  the 
military  which  is  considered  very  important  when  trying  to 
sell  the  service  to  a  potential  recruit.  Reserve  recruiters 
need  experience  in  the  Army  reserve  to  sell  the  reserves  to 
recruits,  and  Active  recruiters  need  experience  in  the 
active  duty  Army  to  sell  it  to  potential  recruits.  Table  10 
summarizes  these  judgments. 

5 .  Demographic  Characteristics 

The  attributes  within  the  Demographic  Characteris¬ 
tics  dimension  are  Age,  Family  Support,  AFQT,  and  College 
Experience.  AFQT  and  Family  Support  were  consistently 
judged  as  most  important  by  both  AGR  and  Active  Recruiters. 
Almost  everyone  said  that  Age  and  College  Experience  did  net 
matter.  The  results  show  this  in  Table  11. 

6 .  Specific  Experience 

Specific  Experience  includes  Sales  Experience, 

Public  Speaking  Experience,  and  Counselling  Experience. 

This  dimension  revealed  significant  variation  between  all 
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TABLE  10 


MILITARY  BACKGROUND  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reserves 


Expert 

1 

2 

4 

5 

6 

Mean 

Most 

YOSlR)  ~ 

Paygrde 

IS  1 

Paygrde 

Paygrde 

Paygrde 

Paygrde 

Important 

.  526 

.846 

mm 

.472 

.  466 

.  561 

.  596 

Least 

YOS(A) 

YOS( A) 

YOS ( A) 

YOS ( A ) 

YOSlR) 

YOS ( A ) 

I mpor  tant 

.227 

.059 

.141 

.  294 

.178 

.097 

Regu 1 ar  Army 


Expert 

7 

8 

9 

10 

11 

12 

1  2 

Most 

Paygrde 

Y0S1 A) 

Paygrde 

YOS  ( A I 

Paygrde 

YOS ( A ) 

Paygrde 

Important 

.44  1 

.456 

.624 

.  566 

.625 

.811 

.  422 

■mm 

Paygrde 

YOSlR ) 

I mpor tant 

.169 

Expert 

14 

15 

16 

Mean 

Most 

Important 

Paygrde 
.  562 

YOS 1  A ) 

.618 

YOS 1  A ) 

.  440 

YOS 1  A ) 

.  498 

Leas  t 

I mpor tan t 

YOSlR) 

.  065 

YOSlR) 

.  112 

Paygrde 
.  229 

YOSlR) 

.041 
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TABLE  11 


DEMOGRAPHIC  CHARACTERISTICS  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reserves 


Expert 

1 

2 

1 

4 

5 

6 

Mean  j 

Most 

AFQT 

Faml ly 

AFQT 

AFQT 

AFQT 

AFQT 

AFQT  i 

I mportant 

.  no 

.  SOS 

.  626 

.  383 

.  381 

.445 

.510  i 

Least 

Co  1  lege 

Age 

Age 

Age 

Age 

College 

Age  ■ 

Important 

.100 

.052 

.02? 

.131 

.048 

.00? 

.029  ! 

i 

Regular  At  nv 


Expert 

7 

8 

9 

10 

11 

12 

13 

Most 

AFQT 

AFQT 

AFUT 

AFQT 

Family 

Faml ly 

Faml ly 

I mpor  rant 

.439 

.414 

.  579 

.  455 

.360 

.  361 

.492 

Least 

Age 

Fami I y 

Faml ly 

Age 

Col  lege 

I mpor  t int 

.03? 

.078 

.014 

.088 

.090 

Expert 

_ _ 

15 

16 

Mean 

Most 

mm 

AFQT 

AFQT 

AFQT 

Impot  tant 

mssm 

.432 

.  707 

.508 

Least 

Co l lege 

Age 

Age 

Age 

I mportant 

.058 

.  132 

.078 

.107 
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three  attributes  as  to  the  order  of  importance.  The 
recruiters  interviewed  felt  that  any  of  the  three  attributes 
help,  but  very  few  recruiters  have  any  kind  of  these 
experiences.  Table  12  displays  the  results. 

D.  COMPOSITE  MODELS 

In  order  to  obtain  composite  models,  two  different 
methodologies  were  used. 

i.  usaa 

Two  separate  mean  models  were  developed,  one 
for  reserves  (MAGR)  and  one  for  active  duty  (MRA) .  To 
develop  this  model,  all  assessments  from  each  expert  were 
sorted  by  concept  and  response,  and  the  means  of  the 
responses  were  calculated.  The  means  were  then  entered  into 
the  expert  system  to  create  a  composite  expert. 

For  both  RA  and  AGR  Mean  expert  systems,  the  overall 
model  Fidelity  Index  was  97,  and  was  at  least  96  for  the 
individual  dimensions.  The  Standards  Index  for  the  overall 
model  was  60  for  the  reserves,  with  the  individual 
dimensions  ranging  from  48.3  to  67.5.  The  actives  Standards 
Index  was  74  for  the  overall  model,  and  ranged  from  46.3  to 
74  in  the  individual  dimensions.  Finally,  the  Discrimina¬ 
tion  Index  for  the  reserves  was  62  for  the  overall  model, 
and  ranged  from  53  to  68  in  the  individual  dimensions.  The 
actives  were  less  discriminatory  with  an  index  of  41  for  the 
overall  model,  and  a  range  of  38  to  65  in  the  individual 
dimensions.  As  expected  the  use  of  mean  input  values 


TABLE  12 


SPECIFIC  EXPERIENCE  DIMENSION 
MOST/LEAST  IMPORTANT  ATTRIBUTES 


Reser  vn 


Expert 

1 

2 

J _ 

4 

5 

6 

Mean 

Most 

S.l  1**3 

Counse  1 

RIR 

Sales 

Counsel 

■ 

Sales 

Important 

.452 

.  569 

RSI 

.  582 

.506 

mm 

.  452 

Least 

Counse  1 

P  .Spk  lnq 

n 

Counsel 

Sales 

P . Spk lnq 

Important 

.239 

.544 

.  210 

.350 

.  285 

mam 

Regular  Army 


Expert 

7 

8 

9 

10 

11 

12 

13 

Most 

Impor  tant 

Counse  1 
.647 

P.Spk lnq 
.453 

Sales 

.470 

P  .Spk 1 nq 

.448 

Counsel 

.375 

Sales 

.715 

Counsel 

.416 

Least 

Important 

P  .  Spk 1 nq 
.153 

Sa  les 
.223 

Counse  1 
.205 

Sa  les 
.261 

Sales 

.313 

Counsel 

.093 

P . Spk 1 nq 
.235 

Expert 

14'  j  15 

16 

Mean 

Most 

I mpor  tant 

P . Spk 1 nq 
.  520 

Counsel 

.488 

Sales 
.  540 

P .  Spk ing 
.330 

L*»ast 

I mpor  tant 

Counse  1 

.232 

Sa  les 
.  197 

Counsel 
.  115 

Counse 1 
.264 
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stabilize  the  disparities  between  individual  experts  by 
creating  an  "average”  expert  system.  The  mean  expert  is 
included  in  Tables  3  through  11  for  comparison  and  summary 
purposes . 

2.  Pa.fcchw,9rfr.Jfodels 

EXPERT87  contains  a  feature  which  allows  the  user  to 
create  a  composite  model  using  the  experts  who  are  logged 
onto  the  system.  The  user  can  patch  experts  to  concepts  or 
dimensions  based  on  any  arbitrary  criteria.  I  have  defined 
the  criteria  using  a  high  fidelity  index  (as  close  to  100  as 
possible) ,  a  normal  (around  50)  Standards  Index,  and  a 
normal  (around  50)  Discrimination  Index.  There  is  no 
"ideal"  criteria.  The  actual  decision  should  be  made  by  the 
user  of  the  system.  However,  for  purposes  of  analysis  and 
based  on  my  judgments,  I  have  defined  the  criteria  as 
stated.  For  example,  Expert  1  meets  the  criteria  for 
Communication  Skills.  That  is,  high  Fidelity,  normal 
Standards,  and  normal  Discrimination  Indices.  However, 
Expert  l's  assessment  on  Personality  Characteristics  is 
below  minimum.  But,  Expert  2  meets  the  criteria  for 
Personality  Characteristics.  So,  the  system  allows  you  to 
patch  the  expert  with  the  concept  and  select  recruiters 
based  on  this  composite  model. 

E.  COMPARING  THE  EXPERT  SYSTEMS 

This  phase  of  the  analysis  used  the  eight  AGR  expert 
systems  (six  experts,  the  mean  model,  and  the  patchwork 
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model),  and  the  twelve  RA  expert  systems  to  evaluate  a  set 
of  twenty  hypothetical  recruiter  applicants.  Subjective 
assessments  were  made  to  determine  measures  on  the 
attributes,  trying  to  make  them  as  realistic  as  possible. 
Many  of  the  attributes  in  the  model  cannot  be  measured 
objectively,  however,  so  an  assumption  was  made  that  the 
ratings  (1)  were  measurable  and  (2)  were  agreed  upon  by  the 
experts  whose  systems  were  used  to  evaluate  the 
"applicants.**  In  other  words,  the  experts  all  assessed  the 
same  twenty  applicants  as  presented  by  each  applicant's 
profile  of  attributes.  The  experts  will  be  compared  by 
their  relative  evaluations  of  the  hypothetical  recruiter 
candidates. 

Table  13  displays  the  profiles  of  the  twenty 
hypothetical  recruiter  candidates.  The  profiles  were 
designed  such  that  some  of  the  applicants  are  at  the  top  end 
of  the  rating  scale  (0-99)  and  some  at  the  bottom  on  all 
attributes.  These  cases  will  illustrate  how  judgments  are 
affected  by  the  Standards  Index.  Experts  who  have  high 
standards  tend  to  assign  lower  ratings  than  more  lenient 
experts. 

The  remainder  of  the  recruiter  applicants  also  have 
specially  constructed  sets  of  attributes.  All  of  the 
applicants  meet  the  minimum  requirements  set  by  the  Army: 

1.  At  least  a  high  school  diploma  graduate  or  GED  with 
one  year  of  college. 

2.  Minimum  GT  score  of  110  waiverable  to  100. 
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TABLE  13 


ATTRIBUTE  RATINGS  OF  TWENTY  HYPOTHETICAL  RECRUITER  APPLICANTS 


TABLE  13  (CONT.) 

ATTRIBUTE  RATI  NOS  OF  TWENTY  HYPOTHETICAL  RECRUITER  APPLICANTS 
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2 
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2 

9 

4 

3 

2 
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fll 

Bl 
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m 
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8 
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fl 
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HI 

9 
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3.  Between  21  and  35  years  of  age. 

4.  In  paygrades  E-5,  £-6,  or  E-7  (E-7's  may  have  no  more 
than  2  years  time  in  grade  at  time  of  selection) . 

The  minimally  acceptable  rating  on  the  scale  used  by  the 

software  is  45.  The  resulting  rankings  of  candidates  is 

contained  in  detail  in  Appendix  D. 

Tables  14  and  15  presents  the  results  of  the  expert 
systems  evaluation  of  the  hypothetical  candidates. 

Asterisks  indicate  the  rejected  applicants  (below  45) .  As 
expected,  the  results  are  very  similar  between  the  AGR 
expert  systems  and  the  Active  expert  systems.  In  almost 
every  case,  the  top  five  applicants  are  B,  L,  E,  C,  and  O. 
Every  case  rejects  A,  K,  and  N  and  they  show  up  as  the  last 
three  applicants.  Applicant  B  was  assessed  at  the  upper  end 
of  the  rating  scale  (all  9's).  The  overall  profile  scores 
for  Applicant  B  range  from  41  to  74.3  in  the  AGR  assess¬ 
ments,  and  from  41  to  83.1  in  the  Active  assessments.  The 
low  rating  makes  those  particular  expert  systems  suspect 
because  they  are  so  contrary  to  the  majority  opinions.  It 
also  makes  no  sense  to  reject  an  applicant  who  is  superior 
in  every  dimension  as  candidate  B  is.  A  review  of  Experts 
12  and  14 's  indices  in  Appendix  C  show  their  expert  systems 
to  be  at  extremes  in  the  Standards  and  Discrimination 
Indices.  The  Mean  expert  for  AGR  recruiters  also  rejects 
applicant  B  with  a  score  of  41  however.  This  is  surprising 
but  may  be  explained  by  the  relatively  low  ratings  overall 
given  by  the  AGR  recruiter  experts.  Results  also  show  that 
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TABLE  14 


EXPERT  fYSTEMS  EVALUATE  HYPOTHETICAL  APPLICAHTS 
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the  expert  systems  created  by  Experts  5,  MAGR,  9,  and  14  had 
the  highest  standards  and  rejected  more  than  half  of  the 
applicants. 

Some  interesting  results  are  in  the  comparisons  of  MAGR 
and  the  patchwork  model  (PAGR) ,  and  MRA  and  the  patchwork 
model  (PRA) .  The  mean  and  patchwork  models  for  AGR 
recruiter  experts  are  quite  different.  Although  both  models 
select  Applicant  E  for  example,  MAGR  ranks  E  as  number  one, 
while  PAGR  ranks  E  number  seven.  However,  MAGR's  rating  is 
60  as  opposed  to  PAGRl's  rating  of  62.8.  This  is  a  good 
example  of  MAGR's  high  standards.  Also  MAGR  rejects 
thirteen  of  the  twenty  applicants  while  PAGR  rejects  only 
four. 

In  contrast,  MRA  and  PRA  are  surprisingly  similar.  They 
both  rank  B  and  L  first  and  second,  respectively.  Similar¬ 
ly,  MRA  and  PRA  reject  applicants  U,  F,  G,  K,  N,  and  A  in 
exactly  the  same  order  with  similar  ratings.  PRA  also 
rejects  H,  T,  and  D  however,  and  MRA  selects  them. 


IV.  CONCLUSIONS  AND  RECOMMENDATIONS 

A.  CONCLUSIONS 

1.  Past  Research 

The  objectives  of  this  thesis  were  to  identify  the 
most  important  factors  and  their  relative  importance  for 
identifying  individuals  with  high  likelihood  of  being 
successful  recruiters. 

An  extensive  literature  review  was  conducted  in  a 
prior  thesis  conducted  by  LCDR  Joyce  Zellweger  in  December 
1986  at  the  Naval  Postgraduate  School,  Monterey,  California. 
It  identified  many  previous  studies  that  examined  the 
recruiter  selection  problem.  Two  distinct  types  of  factors 
have  been  reviewed  for  their  utility  in  predicting 
successful  recruiter  performance.  One  class  of  factors  is 
focused  on  biographical  and  personal  history  variables  which 
can  be  found  in  standard  military  personnel  files.  These 
factors  are  popular  because  they  are  easy  to  identify  and 
measure.  The  biographical  and  personal  history  variables 
most  frequently  used  were  age,  education,  entrance  test 

scores,  gender,  marital  status,  etc.  However, — these  - 

variables  are  extremely  diverse  and  probably  differ  in  terms 
of  the  underlying  dimensions  they  are  intended  to  reflect. 
Knapp  and  Benedict's  1986  study  found  that  the  predictive 
validity  of  individual  variables  contained  within  this 
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category  can  differ  significantly  as  a  consequence  of 
predictors  used  and  the  type  of  criterion  being  predicted. 
Furthermore,  the  vast  majority  of  previous  studies  focusing 
on  a  small  number  of  specific  biographical  or  background 
variables  were  found  to  be  limited  in  their  effectiveness 
when  used  alone  to  predict  sales  success.  [Ref.  16:pp.  19, 
22] 

The  other  class  of  factors  are  given  by  specific 
tests  to  measure  personality  and  behavioral  characteristics. 
Both  sets  of  factors  yielded  disappointing  results. 

Recruiter  characteristics  found  to  be  significant  varied 
across  studies  and  few  results  were  cross-validated  or  could 
not  be  duplicated  upon  cross-validation. 

However,  in  a  1986  study  by  Russell  and  Borman, 
researchers  found  that  vocational  interest  and  personality 
variables  (such  as  dominance,  self-confidence,  and 
spontaneity)  were  significantly  associated  with  military 
recruiter  performance.  Further,  cognitive  variables  such  as 
verbal  ability  and  general  aptitude  appeared  to  have  little 
validity  for  predicting  military  recruiter  success.  They 
also  found  that  although  military  recruiting  is  essentially 
a  sales  job,  the  type  of  "product"  sold  is  quite  different 
from  sales  jobs  in  the  civilian  sector.  Recruiters  sell 
careers  or  life-styles,  not  material  goods  or  services. 
Russell  and  Borman's  study  also  researched  civilian  sector 
sales  jobs  and  found  that  even  though  "product"  type 
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differences  exist  between  the  military  and  civilian  sectors, 
there  are  major  consistencies  between  them  which  contribute 
to  military  recruiter  and  civilian  sales  success.  For 
example,  personality  variables  such  as  dominance  have 
resulted  in  validities  in  both  arenas.  Also,  aptitude  and 
verbal  ability  measures  have  shown  little  merit  for 
predicting  either  recruiter  performance  or  civilian  sales 
success.  Skill  level  variables,  such  as  assessment  center 
scores,  may  be  useful  predictors  of  recruiter  job 
performance.  [Ref.  17:pp.  vi-vii) 

In  most  prior  studies,  the  criterion  problem  was 
probably  the  single  most  reason  why  relatively  little 
variance  in  recruiter  productivity  was  explained.  The  poor 
findings  were  usually  a  result  of  the  failure  to  properly 
conceptualize  predictor-criterion  relationships.  Objective 
criterion  measures  usually  reflect  the  results  of 
— salesperson  behavior  and  environmental  factors.  [Ref.  16 :p. 
33] 

2.  Expert  .Systems 

This  thesis  applied  a  relatively  new  methodology  to 
the  recruiter  selection  problem—Expert  Systems.  Expert 
Systems  is  a  field  of  Artificial  Intelligence  which  has  been 
particularly  successful  in  solving  the  types  of  problems 
where  there  is  incomplete  knowledge  and  where  judgment  is 
needed. 
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The  expert  system  shell  utilized  in  this  thesis  is 
called  EXPERT87 .  It  is  a  special  type  of  expert  system 
intended  for  the  large  class  of  moderately  difficult  and 
repetitive  decision  problems  which  so  often  face  managers. 

It  allows  efficient  interaction  of  experts  with  a  knowledge 
base  and  develops  a  model  using  one  of  more  experts' 
knowledge,  judgments,  experience,  and  intuition  to  solve 
problems. 

EXPERT87  also  addresses  the  problem  that  studies 
using  personality  and  behavioral  characteristics  have  not 
been  able  to  address  of  revealing  the  relative  importance  of 
successful  recruiter  attributes  in  relation  to  each  other. 
Since  it  does  not  rely  on  an  objectively  measured  criterion, 
this  approach  also  avoids  the  problem  of  poorly  specified 
and  measured  performance  criteria  that  has  plagued  much  of 
the  previous  empirical  modeling  efforts  to  profile 
successful  recruiters. 

3.  Profile  of  the  Successful  Recruiter 

The  model  developed  for  this  thesis  was  based  on 
characteristics  previous  studies  found  to  be  related  to 
recruiter  success  as  developed  in  LCDR  Zellweger's  thesis, 
and  then  refined  to  obtain  a  more  efficient  set  of 
attributes.  These  attributes  were  organized  into  larger 
dimensions  of  the  hierarchy. 

Six  Army  reserve  and  ten  active  duty  Army  recruiters 
evaluated  the  model,  and  EXPERT87  estimated  their  expert 
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systems.  In  addition,  I  developed  two  composite  experts 
using  the  means  of  all  assessments  made  by  the  experts. 

This  allowed  me  to  develop  a  composite  model  and  compare 
similarities  and  differences  with  the  individual  models.  I 
also  used  a  patchwork  model  (which  is  part  of  the  EXPERT87 
software)  to  develop  alternative  composite  experts.  The 
results  were  very  similar  as  to  which  characteristics  were 
more  important  than  others.  The  AGR  recruiters  felt  that 
Communication  Skills,  Demographic  Characteristics,  and 
Personality  Characteristics  were  most  important  in  a 
successful  recruiter.  The  active  recruiters  felt  that 
Communication  Skills,  Personality  Characteristics,  and 
Behavior  Characteristics  were  most  important.  Within  these 
dimensions,  the  most  important  attributes  were  Integrity, 
Listening,  Informing,  Persuading,  AFQT,  Family  Support, 
self-starter,  and  Extroverted.  The  mean  models  also  support 
these  results. 

B.  RECOMMENDATIONS 

1.  Measure  Important  Attributes 

Personality  and  Behavior  characteristics  come  up 
time  and  time  again  as  one  of  the  top  concepts  related  to 
successful  recruiters.  Described  below  are  a  variety  of 
measurement  instruments  designed  to  measure  personality, 
a.  Myers-Briggs  Type  Indicator  (MBTI) 

Isabel  Briggs  Myers  and  Katharine  C.  Briggs 
developed  this  test  which  consists  of  four  dichotomous 
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indices  of  personality  types:  Extraversion-Introversion 
(El),  whether  perception  and  judgment  are  directed  toward 
the  environment  or  the  world  of  ideas;  Sensation-Intuition 
(SN) ,  indicating  dominant  perceptual  style;  Thinking-Feeling 
(TF) ,  which  one  of  these  two  modes  of  judgment  is  relied 
upon;  and  Judgment-Perception  (JP) ,  indicating  which  of 
these  is  relied  upon  in  dealing  with  the  environment.  The 
test  consists  of  166  forced-choice  (usually  two)  items. 
Fifty-two  items  are  word  pairs  in  which  respondents  indicate 
a  preference.  Some  of  the  pairs  are  theory-certainty, 
build-invent,  casual-correct,  who-what,  sign-symbol  or 
similar  to  the  following: 

Do  you: 

(1)  prefer  to  do  things  at  the  last  minute 

(2)  find  it  hard  on  your  nerves 

The  test  is  self-administering  and  has  no  time  limit,  but 
usually  takes  about  50  minutes  to  complete.  The  MBTI  is 
easy  to  administer  and  score,  and  the  types  do  have  the 
virtue  of  being  mutually  independent.  A  draw-back  for  our 
purposes  is  that  it  only  measures  a  couple  of  the  attributes 
identified  (extrovert  and  self-image) .  It  is  available 
through  the  Educational  Testing  Service,  Princeton,  New 
Jersey  08540.  (Ref.  18:pp.  186-189] 

b.  California  Psychological  Inventory  (CPI) 

The  CPI  was  developed  by  Harrison  G.  Gough.  It 
groups  eighteen  variables  under  four  classifications:  Class 
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I  measures  poise,  ascendancy,  and  self-assurance;  Class  II 
measures  socialization,  maturity,  and  responsibility;  Class 
III  measures  achievement  potential  and  intellectual 
efficiency;  and  Class  IV  measures  personal  orientation  and 
attitudes  toward  life.  This  one  test  measures  most  of  the 
attributes  identified  in  the  expert  system  approach  to 
profiling  the  successful  recruiter.  Specifically,  it 
includes  measures  of  self-starter,  extroverted,  people- 
oriented,  self-image,  flexibility,  commitment,  and 
indirectly,  integrity.  Integrity  could  be  measured  using 
the  variables,  responsibility  and  socialization.  They  are 
defined  by  the  CPI  as  follows: 

(1)  responsibility — indicating  seriousness  of 

thought  and  manner,  conscientiousness,  dependability 
and  uprightness;  being  the  kind  of  person  that  others 
tend  to  trust  and  to  rely  upon. 

(2)  socialization — indicating  a  strong  sense  of  probity 
and  propriety;  acceptance  of  rules,  proper  authority, 
and  custom;  a  person  who  seldom  if  ever  gets  into 
trouble. 

The  CPI  is  essentially  self-administering  and 
consists  of  480  statements.  The  18  scales  are  normative  and 
are  based  on  over  6,000  males  and  7,000  females.  The  raw 
scores  are  converted  to  profiles  which  provide  graphic 
representations  of  standard  scores. 

Convincing  evidence  exists  to  validate  each  of 
the  18  scales.  Even  attributes  such  as  self-acceptance 
revealed  significant  differentiation  between  high  school 
students  rated  as  high  and  low  on  self-acceptance  by  staff 
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assessment  ratings.  In  test-retest  reliabilities  reported, 
high  school  subjects  were  tested  after  one  year.  The  median 
test-retest  correlation  was  .65  for  males  and  .68  for 
females.  [Ref.  18:pp.  37-40] 
c.  The  16PF 

The  16PF  is  a  personality  test  designed  to 
measure  an  individual's  personality  in  terms  of  sixteen 
basic  factors.  It  was  used  successfully  in  a  predictor 
battery  in  a  Marine  Corps  Study  conducted  by  Larriva  (1975). 
Some  of  the  factors  measured  which  have  been  associated  with 
recruiter  success  were  dominance,  aggressiveness,  self- 
confidence,  and  spontaneity. 

2.  Test  the  Model 

Before  making  any  further  modifications  to  the 
model,  field  test  it.  An  appropriate  expert  system  could  be 
based  on  criteria  set  forth  by  the  Recruiting  Command  and 
input  values  measured  for  the  attributes  of  selected 
recruiter  candidates  at  entry  to  recruit  training  school. 
These  recruiters  could  be  tracked  for  at  least  one  year  to 
determine  their  performance  and  the  validity  of  the  model  to 
detect  potentially  unsuccessful  recruiters.  The  results  of 
the  validity  analysis  could  then  be  used  to  modify  the 
existing  model . 

3 .  Improve  the  Model 

Results  of  this  analysis  and  suggestions  from  the 
experts  who  participated  in  this  project  indicated  that  some 
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of  the  attributes  in  the  model  may  be  eliminated  without 
affecting  recruiter  selection.  In  other  words,  they  are  not 
important  to  recruiter  success.  The  attributes  identified 
as  unimportant  were  writing  skills,  age,  sense-of-humor,  and 
decisiveness.  By  giving  these  attributes  a  weight  of  one, 
the  model  could  be  tested  to  see  if  the  results  of  the 
twenty  hypothetical  recruiters  remains  the  same.  If  they 
do,  these  attributes  could  be  eliminated  and  probably  not 
affect  the  model. 

4.  Work  Remaining 

Many  possible  military  personnel  selection 
applications  exist  for  this  type  of  methodology.  This  study 
revealed  very  few  differences  between  reserve  and  active 
duty  recruiters  in  their  perceptions  of  what  makes  a 
recruiter  successful.  This  same  model  could  be  applied  to 
the  active  Army  in  selecting  recruiters  and  could  be  tested 
across  the  services  to  determine  whether  there  are 
significant  differences  in  their  perceptions  and  knowledge 
of  what  characteristics  define  a  successful  recruiter.  This 
methodology  could  be  applied  to  any  type  of  personnel 
selection  problem  where  consistent  and  objective  decisions 
must  be  made  and  objective  performance  criteria  are 
difficult  to  measure. 
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ATPENDIX  A 


Author  : 
Title-: 

2ki_L 


ANNOTATEL'  H  f  BL  1  OtJRAPHy 


Department  of  the  Army  (July  1985) 

Acs  ion  merit  r,  f  Enlisted  Per sonne  1  to  the  U  ■  S  . 

Ar  my  Rocr  i  i  1 1  no  Comma  nd 

To  prescribe  the  procedures,  criteria,  and 
personnel  actions  required  for  the  selection  and 
assignment  of  Rngular  Army  and  Reserve  enlisted 
personnel  for  service  as  U.  S.  Army  recruiters; 
it  outlines  the  policy  concerning  selection  and 
assignment  of  enlisted  personnel  to  USAREC 
administrat ive  support  positions;  and  it 
prescribes  the  management  policies  applicable  to 
all  enlisted  personnel  while  assigned  to  USAREC. 


Toquam,  J.  L.,  and  Ro3se,  R.  L., 


Author :  Dorman,  W.  C., 

(May  1979) 

Title:  An  T  nventor  v  Battery  to  Predict  Navy  and  Marine 

Corps  Recruiter  Per  forma  nee :  De ve 1 opment  and 
Va  1 ida  t i on 

Oh  j :  To  develop  penci 1 -and-pen  predictors  of  Navy 

and  Marine  Corps  recruiter  effectiveness  and 
evaluate  the  validity  of  these  measures. 

Method :  A  trial  predictor  battery  of  personality, 

interest,  and  biographical  items  and  scales 
was  developed  based  on  a  literature  review  of 
previous  military  recruiter  selection  studies. 
The  battery  was  given  to  a  geographical ly 
representative  sample  of  329  Navy  and  113 
Marine  Corps  recruiters.  Scores  on  the 
predictor  measures  and  performance  ratings  were 
correlated.  An  objective  effectiveness  index 
was  also  used  in  a  concurrent  validation 
design.  The  relationships  between  performance 
criteria  and  the  various  predictors  were 
'ssessed  . 

Cone  1 :  a.  In  the  Navy  sample,  the  estimated  cross¬ 

validities  for  predictor  composites  against 
four  of  the  five  performance  criteria  were 
significantly  different  from  zero  at  the  .01 
level.  (Ranged  from  .17  to  .31) 

b.  In  the  Marine  Corps  sample,  validity 
estimates  ranged  from  .22  to  .38,  with  all  five 
predictor  composite-performance  criterion 
relationships  significantly  different  from  zero 
at  the  .05  level . 

c.  This  predictor  battery  shows  promise  for 
helping  the  Navy  and  Marine  Corps  decision¬ 
makers  in  selecting  recruiters. 

Stren :  a.  The  researchers  went  to  great  lengths  to 

arrive  final  performance  criteria  that 
reflected  relevant,  reliable,  and  comprehensive 
measures  of  recruiter  performance.  As  a 

-  result,  reasonably  good  validities  were 

obtained  in  the  study. 

b.  Controlled  for  the  fact  that  subjective 
performance  evaluations  are  sometimes  unreli¬ 
able  by  paying  careful  attention  to  defining 
performance  dimensions  and  selecting  the  proper 
persons  to  provide  the  ratings,. 


Wea  k  : 


a.  The  cross-val  Idi  ty  coefficients  computed 
ate  only  estimates  of  the  composites' 
predictive  validity. 

b.  The  admin istrat i ve  net  for  persons  in  this 
study  are  probably  different  than  the  kind  of 
set  they  would  have  had  if  they  were  actually 
taking  the  inventories  as  applicants  before 
being  accepted  for  recruiting  duty. 

c.  Rating  errors  of  leniency,  restriction  of 
range,  and  hale  are  evaluated  only  indirectly. 
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Author  ; 


4  . 

Title: 


'Ab±: 


ho. 


'.■jno  1 


We  3  k  : 


Pep .  V: 


Elig,  T.  W . ,  Gade ,  P.  A.,  and  Johnson,  R.  M . , 
(undated) 

Recruiter  and  Recruit  Demouraph ic 

Cha  r ac t e r i ?  1 1 eg :  A  F r e I  1  mi na r  v  I nve s t i  l  j- 

t  i  on  of  Recr_u  iter  Se lect  i  on  Cr  iter  ia 

To  describe  a  new  approach  to  recruiter 

select  ion  research. 

Utilises  the  Military  Enlistment  Processing 
Station  Reporting  System  (MRS)  to  develop 
measures  of  recruiter  performance  based  on 
recruit  characteristics.  Or.e  source  of  data 
used  was  the  Army's  Enlisted  Master  File  ( EMF ) 
A  sample  of  612  was  selected  based  on  three 
concerns:  a.  identify  production 

recruiters  Erom  non-pr oduct i cn  personnel, 
b.  The  recruiters  must  have  sufficient  time 
as  a  full  production  recruiter  at  the  same 
location  to  help  control  for  opportunity  bias 
and  seasonality  fluctuations,  and  c.  The 
sample  should  contain  a  sufficient  number  of 
females  to  do  cross  gender  comparisons. 

The  analysis  falls  into  two  categories: 

a.  Demographic  and  background  character ist ics 
of  recruiters  were  systemat ical ly  explored 
through  descriptive  statistics. 

b.  Recruiter  personnel  character ist ics  were 
systemat lea  1 ly  correlated  with  contract 
production  and  identified  through  analysis  of 
covariance  techniques. 

This  study  demonstrates  that  recruiter  demo¬ 
graphic  character  istics  can  be  related  to 
recruit  character i st ics  when  opportunity  bias 
is  removed.  The  best  Army  recruiters  were 
found  to  be  better  educated,  have  a  higher 
AFQT,  and  if  male,  younger,  and  if  female, 
older.  Thus,  it  appears  that  demograph i c ’ da ta 
may  be  useful  for  selecting  recruiters  from  a 

non-voluntary  pool.  .  . . . .  . . . . 

Controlled  for  opportunity  bias  and  identified 
some  interesting  attributes  of  successful 
recruiters . 

The  small  sice  of  the  sample  resulted  Ln  many 
variables  not  able  to  be  cross-validated  to 
test  for  interaction  effects  ( par t icular ly 
gender).  There  were  only  60  women  recruiters 
in  the  data  base . 

The  number  of  applicants  contracted  in  FY79 
adjusted  by  the  covariate,  DRCAVG  (opportunity 
bias  )  . 


Aodep .  V :  gender,  education  and  AFQT  (renormed), 

recruiter's  age,  recruiter's  ethnic  group 
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Author : 
Title: 


Ob  j  : 


Method : 


Cone  1 : 


Veak  : 


Pep  .  V.  : 

I ndep . V : 


Brown,  G.H.,  Wood,  M.D.,  and  Harris,  J.  D. 

( May  1978) 

Army  Recruiters:  Criterion  Deve 1 opment  and 
Prel lminary  Va I i da t Ion  of  a  Select  Ion 
Procedure 

To  develop  a  valid  criterion  of  recruiter 
effectiveness  and  develop  a  test  battery  to 
identify  those  most  likely  to  succeed  as 
r  ecr ul ters • 

Stepwise  Multiple  regression;  Benchmark 
Achievement  Scores  (BAS);  Simple  Achieve¬ 
ment  Scores  ( SAS ) . 

BAS  Score  =  (Actual  Pr oduct ion/Pr ed ic ted 
Production)  X  100 

SAS  Score  -  (Total  Production/Average 
Production  in  DRC)  X  100 
■a.  50%  of  the  variance  in  production  scores 

derives  from  factors  unrelated  to  individual 
recruiter's  character istics. 

b.  Simple  Achievement  Scores  (SAS)  appear  ■ o 
be  a  more  equitable  measure  of  a  recruiters 
effectiveness  than  other  more  traditional 
measures . 

c.  Twenty  background  items  that  may  be  useful 
were  identified  for  selecting  recruiters. 

a.  Territorial  information  does  not  refer  to 
the  specific  geographical  area  in  a  recruiter’s 
area  because  the  information  is  not  available. 

b.  Unemployment  rate,  median  family  income, 
and  education  level  of  community  came  from  the 
U.S.  Census  Bureau  and  data  is  organized  only 
by  county  or  city--many  recruiting  territories 
comprise  portions  of  several  counties. 

c.  Administering  the  Test  Pattery-- 
administered  at  two  locations,  one  a  busy 
recruiting  station  and  the  other,  a  group 
testing  situation  was  not  achieved.  The 
ultimate  effect  could  not  be  ascer ta i n-d . 

Total  Production 

Average  production  per  recruiter  in  subject's 
District  Recruiting  Command  (DRC);  Average 
market  share  for  station  zone;  proport. ?n  of 
zone  suburban;  months  of  recruiter  experience; 
number  of  high  school  seniors  in  zone;  average 
production  per  recruiter  for  subject's  Regional 
Recruiting  Command  (RRC);  number  of  AS'-'AB's  in 
subject's  DRC;  number  of  17-21  year  olds  in 
college  in  station  zone;  size  of  station  zone 
in  square  miles;  proportion  of  zone  rural; 
proportion  of  zone  metro;  ratio  of  qualified 
military  available  (QMA)  to  military  available 
(MA). 
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Author  : 


Title; 


Method 


Concl : 


Borman,  W.C.,  Hough,  L.M.,  and  Dunnette,  M.D. 
(Feb.  1976) 

Development  of  Behaviorallv  Baaed  Rat  ino  Scales 
for  Evaluating  the  Per  f ormance  of  U . S .  Navy 
Recruiters 

To  develop  and  field  test  performance  rating 
scales  for  measuring  Navy  recruiter  Job 
effectiveness  -  first  phase  of  research. 

Used  behavior  scaling  methodology  to  gather 
800  critical  incidents  describing  different 
facets  of  effective  and  ineffective  recruiter 
performance  from  field  recruiters,  their 
superiors,  and  recruits.  These  incidents  were 
classified  into  nine  dimensions:  a.  Locating 
and  contacting  qualified  prospects,  b.  Gaining 
and  maintaining  rapport,  c.  Obtaining  informa¬ 
tion  from  prospects  and  making  good  person-Navy 
fits,  d.  Salesmanship  skills,  e.  Establishing 
good  relat ionships  in  the  community,  f.  Provid¬ 
ing  knowledgeaole  and  accurate  information 
about  the  Navy,  g.  Administrative  skills, 
h.  Supporting  other  recruiters  and  the  command, 
and  i.  Dedication  to  the  job. 

In  addition,  a  different  group  of  recruiters 
made  similarity  judgements  between  every 
possible  pair  of  a  subset  of  60  behavior 
examples  randomly  chosen  from  a  large  pool  of 
incidents.  The  results  were  analyzed  using 
nonmetric  multidimensional  scaling  (MDS). 
Regression  analysis  was  used  to  define  the 
pattern  of  contributions  made  by  various  com¬ 
binations  of  the  MDS  dimensions  to  each  of  the 
retranslation  dimensions.  MDS  dimensions  were 

a.  Gathering  information  about  applicants, 

b.  Planning  and  organizing  recruiting 
practices;  looking  ahead  to  future  recruiting 
requirements,  c.  Expending  extra  effort  to  aid 
applicants  or  recruits,  d.  Salesmanship; 
listening  to  the  prospect  and  then  making  an 
appropriate  and  effective  sales  pitch,  and 

e.  Expending  extra  effort  related  to  prospect¬ 
ing  activities. 

Identified  more  than  800  critical  incidents 
describing  different  facets  of  effective  and 
ineffective  recruiter  performance.  Results  of 
the  field  test  were  analyzed  and  revealed  that 
self  and  peer  ratings  contained  impressive 
convergent  and  discrimanant  validity. 
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«***  ■■ 


Stren : 


Weak: 


Da  ta : 


Met  objective  successfully.  Good  first  step  in 
research.  The  rating  scales  provided  an  oppor¬ 
tunity  to  assess  the  validity  of  procedures 
presently  being  used  and  those  developed  in  the 
future  to  select  individuals  to  recruiting 
duty.  Useful  to  "educate”  Commanding  Officers 
and  persons  considering  recruiting  duty. 
Restricted  to  self  and  peer  ratings  to  assure 
highest  reliability  and  valid  performance 
appraisals.  Supervisory  ratings  are  used  to 
portray  overall  job  effectiveness  of  each 
recruiter.  Data  was  insufficient  to  conclude 
unequivocally  that  the  rating  scale-rating 
source  assignments  would  always  be  optimal  in 
obtaining  valid  performance  indices  for 
recru i ters . 

The  fourteen  variables  described 
"method"  section  above  were  used 
tlonal  studies  comparing  the  two 
dimensions  (Behavior  scaling  and 


in  the 

for  correla- 
set3  of 
MDS)  . 
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Author : 


and  Rosse,  R . L . 


Title: 
O-bj  ■  J 

Method : 
Co nc 1 : 

Streng : 


Borman,  V.C.,  Toquam,  J.L., 

(March  1977) 

Dimens  ions  o f  the  Army  Recru i ter  and  Guidance 
Counselor  Job 

Determining  performance  requirements  of  the 
recruiter  job  by  defining  underlying  task 
dimensions  associated  with  recruiter  and 
guidance  counselor  jobs. 

Multidimensional  Scaling  (MDS)  and  a  clustering 
procedure 

A  composite  list  of  four  broad  dimensions  (Pros¬ 
pecting  activities.  Publicizing  the  Army, 

Selling  the  Army,  and  Administrative  Activities) 
was  formed  and  could  be  useful  in  developing 
selection  procedures  for  potential  Army 
recruiters. 

The  MDS  solution  is  a  general  framework  for 
representing  the  four  broad  dimensions  and  the 
clustering  solution  defines  the  MDS  dimensions 
more  specifically.  Could  be  useful  in  identify¬ 
ing  attributes  necessary  to  succeed  in  a 
recruiter  job. 

Did  not  identify  personal  characteristics  and 
attributes  of  successful  Army  recruiters. 


Weak : 


8.  Author :  Graham,  W.R.,  Brown,  G.H.,  King,  W.L.,  and 
“  Wood,  M.D.  (March  1973) 

Title :  A  Pilot  Study  of  Army  Recrui  ters :  The i r  Job 

Behaviors  and  Personal  Character ist ics 
Obi :  Pilot  study  to  develop  hypotheses  concerning 

the  character ist  ics  and  job  behaviors  associ¬ 
ated  with  recruiter  success. 

Method :  Interviews  of  79  recruiters  including  subjects 

with  high,  average,  and  low  records  of  success, 
in  terms  of  percentage  of  quota  achieved. 

Cone  1 :  a.  Based  on  sel f -descr ipt ion  data,  few 

characteristics  were  significantly  related 
to  production  records. 

b.  High  producers  were  less  likely  than  low 
producers  to  cite  "independence"  as  a 
source  of  job  satisfaction. 

c.  High  producers  were  less  likely  than  low 
producers  to  complain  about  long  hours  of 
work . 

d.  High  producers  more  often  than  low 
producers  mentioned  the  use  of  Pre- 
Induction  Physical  (PIP)  cards  and  mail- 
outs  as  prospecting  techniques  they  had 
found  successful. 

e.  High  producers  less  often  admitted 
communication  problems. 

f.  High  producers  were  less  likely  to  describe 
themselves  as  "not  irritable"  and  "empathe- 
tic". 

Stren ;  Good  first  step  for  further  research. 

Weak :  Successful  in  meeting  its  principal  objective, 

but  findings  suggest  that  local  situational 
factors  may  have  such  Impact  as  to  preclude  any 
simple  relationship  between  selection  variables 
and  criterion  performance.  Small  sample  size, 
not  representat ive--all  79  recruiters  were  from 
the  3rd  Recruiting  District. 
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Author : 
Title: 
Obi  : 

Method : 
Cone  1 : 


Stren : 


Weak  : 


Hirabayashi,  D.,  and  Hersch,  R.S.  (Dec.  1985) 
Excellence  in  Navy  Recruiting 
To  document  characteristics  of  excellent  Navy 
Recruiting  Districts. 

Interviews 

Navy  recruiters  are  movers,  shakers,  and  sales¬ 
men;  hungry  for  success/promotion;  aggressive, 
want  responsibility  and  to  excel;  have  outstand¬ 
ing  communication  skills,  fundamental  knowledge 
of  recruiting,  inherent  skill  with  numbers, 
sales,  and  the  public;  ambitious,  extroverted, 
and  like  to  meet  and  talk  to  people;  positive, 
cheerful,  enthusiastic,  and  self-motivated. 
Although  not  useful  for  empirical  analyis,  could 
be  great  information  for  expert  systems. 

Did  not  develop  a  model  -  was  not  tested.  This 
study  was  based  only  on  interviews. 


Author  : 
Title: 

Obi: 

Method: 


Concl : 


Streng : 

Weak  : 


Wollack,  L.,  and  Kipnis,  D.  (March  1960) 

Deve 1  cement  o t  a  Device  for  Selecting 
Recrui ter  s 

To  develop  a  Navy  recruiter  selection 
battery  and  conduct  a  concurrent  validity 
study  to  assess  the  battery's  usefulness  as 
•  »  -.screening  device.  (Navy  study) 

The  predictor  battery  was  composed  of  13 
measures  designed  to  reflect  fluency  of 
expression,  knowledge  of  the  Navy,  interest 
in  recruiting  activities,  and  general  aptitude. 
The  validation  sample  consisted  of  410  active 
duty  recruiters  representing  40  main  recruiting 
stations,  substations,  and  branch  stations. 

The  sample  was  formed  by  calling  the  commanding 
officers  and  asking  them  to  nominate  the  most 
and  least  effective  recruiters  from  their 
respective  stations.  The  nominations  were 
used  as  the  criterion  measure  against  which  the 
predictors  were  validated. 

The  sample  used  to  cr oss-val idate  the  predictor 
battery  consisted  of  260  students  attending  the 
6-week  recruiter  course. 

Willingness  to  work  and  recommendation  for 
recruiting  duty  correlated  significantly  (p  < 
.01)  with  Technical  competence.  The  only  other 
predictors  which  correlated  significantly  with 
any  of  the  criteria  was  fluency-of -expression 
which  yielded  a  correlation  of  -.18  (p  ,  .01) 
with  Military  manner.  Overall,  the  results 
of  the  cross-validation  analysis  indicated  that 
a  successful  recruiter  has  persuasive 
interests,  Is  not  overly  interested  in 
scholarly  pursuits,  and  believes  in  the  value 
of  a  Navy  career . 

The  predictor  battery  contains  a  variety  of 
tests  and  inventories  with  reasonably  high 
reliability  of  the  criterion  nominations. 

Only  a  limited  number  of  the  scales  and  item 
keys  cross-validated  significantly.  This  is 
probably  due  to  the  fact  that  the  raters  were 
making  their  evaluations  on  the  basis  of 
reputation  instead  of  performance  or  because 
many  of  the  individual  differences  that  are 
predictive  of  recruiter  success  were  not 
included  in  the  battery. 


11.  Author :  Massey,  I.  M.,  and  Mullins,  C.  J.  (Feb.  1966) 

Title:  Va  1  i  da  1 1  on  n  f  t.h>.-  Per-gu  i  to  r  -Salesman  3 e  1  e c  1 1  o n 

Tost 

Ob i :  To  validate  the  Recrui ter  -Sa I esman  selection 

test . 

M ; t hod :  An  eight  inventory  battery  was  developed  to 
measure  qualities  such  as  empathy,  friendli¬ 
ness,  sociability,  and  perseverance .  Predictor 
variables  were  correlated  with  school  success 
and  supervisor  field  ratings. 

Cone  1 :  Cross-val idation  results  indicated  that  the 

battery  would  be  useful  only  marginally  in 
predicting  school  performance  and  not  at  all  in 
predicting  field  ratings. 

Weak :  Supervisor  rating  criterion  probably  caused  the 

poor  results  because  of  "halo"  and  "leniency" 
effects . 


12. 


Author: 

Title: 


Ob  j  : 
Concl : 


Stren : 

Weak  : 


Pep  .  V .  : 
Explan : 


Krug,  S.E.  (Nov.  1972) 

Psychological  and  Demographic  Predictors  of 
Success  as  a  Naval  Recruiter 
To  develop  a  personality  test  which  would  be 
useful  in  predicting  sales  ability. 

The  typical  effective  Navy  recruiter  is 
married,  has  more  years  of  formal  education, 
and  tends  to  be  warm,  outgoing,  dominant, 
aggressive,  self-assured,  and  has  relatively 
conservative  political  views.  This  battery  was 
used  to  screen  people  for  recruiting 
assignments  between  1972  and  1976. 

Identified  possible  attributes  of  a  successful 
recruiter;  cross-validation  was  performed  and 
indicated  the  equation  would  be  useful. 

Actual  use  proved  the  equation  did  not  predict 
sales  ability  effectively. 

Success  as  a  Naval  Recruiter 

Marital  Status;  Years  of  Education;  X3  (score 
range  of  0-21);  X4  (score  range  of  0-5);  X5 
(score  range  of  0-11);  Warm/Aloof;  Dominant/ 
Submissive;  Apprehensive/Self-assured;  Experi¬ 
menting/Conservative;  MD  (score  range  of  0-14) 
(First  description  before  slash  is  descriptive 
of  higher  scores,  above  5.5;  below  slash,  lower 
scores . ) 
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13 .  Author  : 
Title: 


Method : 


Ccncl : 


Stren : 


Weak  : 


Pep.  V.  : 

Exp.  V.  : 


Shupack,  M. A.  (June  1979) 

An  Analys is  of  the  Cost  Implications  of  Employ  - 
ing  Success  Predictive  Cr i ter ia  in  the  Process 
of  Select i nq  Navy  Recr u i te r s 
To  identify  success  predictive  criteria  for 
selection  of  Navy  recruiters  using 
NAVCRUITCOM ' s  Honor  Roll  as  MOE.  Also,  to  try 
and  explain  the  variance  in  recruiter 
performance . 

Multiple  Regression  Analysis  on  the  entire 
sample  group  as  well  as  three  subgroups: 

ALLMED I OCRE  -  personnel  who  completed  the 
entire  20-month  test  period. 
ALLSUCCESS  -  personnel  who  completed  the  test 
period  and  achieved  the  MOE. 
ALLDROP  -  those  who  failed  to  complete  test 
period. 

Test  period  -  Jan.  1,  1977  -  Aug.  30,  1978 
Sample  -  1262  male,  active  duty  canvasser 
recruiters. 

A.  The  character istics  which  contributed  most 
to  the  explanation  of  variance  for  the  success¬ 
ful  recruiters  (ALLSUCCESS)  were  education, 
paygrade,  and  enlisted  entrance  test  scores. 

For  unsuccessful  recruiters  (ALLDROP),  the  best 
predictors  were  rate,  active  duty,  and  enlisted 
entrance  test  scores. 

B.  Successful  recruiters  were  most  often  in 
paygrades  E5,  E6,  had  6-14  years  of  active 
duty  service,  GAM  scores  between  130-150,  SAM 
scores  between  65-80,  and  High  School  diploma 
or  up  to  two  years  of  college.  No  strong 
trends  regarding  rate  were  distinguishable. 

C.  Recruiter  attrition  has  substantial  finan¬ 
cial  implications.  245  personnel  were  in  the 
ALLDROP  group  of  this  study,  costing  almost 
three  million  dollars  in  base  pay,  BAQ,  and  PCS 
costs  alone. 

Reasonably  good  identification  of  factors  which 
are  indicative  of  success  and  failure. 

None  of  the  statistical  analysis  identified 
characteristics  of  the  mediocre  group 
( ALLMED I OCRE ) ;  all  values  for  all  variables  for 
all  individuals  in  the  sample  were  not 
obtained,  resulting  in  missing  values. 

Honor  Roll  Performance  (the  number  of  times  an 
individual  recruiter  appeared  on  the 
NAVCRUITCOM  Honor  Roll  during  the  test  period) 
paygrade;  education;  active  duty  (number  of 
years);  General  Comprehension  Test,  Arithmetic 
and  Mechanical  Score  (GAMX)  -  on  Navy's 
enlisted  entrance  test;  Sales  Aptitude 
Battery/16PF-m  test  score  (SABX);  rate;  age. 
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Author : 
Title: 
Obi : 


Method : 


Concl : 


Stren : 
Weak : 


Arima,  James  K.  (April  1976) 

A  Systems  Analysis  of  Navy  Recruiting 
To  investigate  and  document  Navy  recruiting 
as  a  process  that  interacts  with  the  larger 
military  community  of  which  it  is  a  part  and 
the  civilian  community  which  provides  the  raw 
materials  it  processes  into  accessions  for  the 
Navy. 

Interviews  were  conducted  and  operations  were 
observed  at  every  echelon  of  the  Navy  Recruit¬ 
ing  Command  (NAVCRUITCOM),  Armed  Forces 
Entrance  and  Examining  Stations,  the  Armed 
Forces  Vocational  Testing  Group,  and  the 
Recruit  Training  Command  during  the  last  three 
quarters  of  fiscal  year  1974. 

This  study  focused  on  how  the  Navy  Recruiting 
Command  operates  and  made  recommendations  to 
improve  the  system.  The  only  result  identified 
that  related  to  this  thesis  was  that  failures 
in  recruit  training  were  attributable  to  faulty 
selection  by  NAVCRUITCOM.  Furthermore,  goal 
accomplishment  for  the  fiscal  year  was 
accomplished  by  failing  to  meet  the  quality 
standards  that  were  in  effect.  The  prevailing 
norm  in  the  system  was  dedication  to  hard  work 
in  order  to  meet  recruiting  goals. 

Documents  recruiter  problems. 

No  direct  application  to  the  problem  of 
profiling  the  successful  recruiter. 


Author  : 
Title: 


Method : 


Concl : 


Deo .  V: 


Indep .  V: 


Larriva,  R.  F.  (1975) 

”U.  S.  Marine  Corps  Recruiter  Performance 
Prediction  Study" 

To  apply  the  Navy's  recruiter  selection 
battery  to  a  Marine  Corps  sample. 

The  inventory  was  administered  to  all  active 
duty  recruiters  from  the  9th  Marine  Corps 
district  ( N  -  <1 7  0 )  and  consisted  of  the  same 
items  used  in  the  Krug  (1972)  study.  The 
author  analysed  the  data  using  the  formula 
developed  by  the  Navy  to  predict  recruiter 
success . 

The  analysis  indicated  that  the  Navy  formula 
was  not  a  valid  equation  for  predicting  Marine 
Corps  recruiter  production.  However,  wuen 
recruiting  setting  (rural-urban)  and  geographi 
cal  differences  were  taken  into  account  in  the 
criterion,  relatively  successful  prediction 
formulas  were  formed,  one  for  urban  recruiters 
and  one  for  rural  recruiters.  Cross-va 1  id i ty 
results  suggest  that  this  method  may  be  a 
useful  screening  device  for  candidates  for  the 
Marine  Corps  recruiter  job. 

The  number  of  accessions  of  urban  and  rural 
recruiters  separately  and  corrected  for 
geographical  differences  in  relative  perfor¬ 
mance  of  recruiters. 

Urban,  Urban  Hi-Low,  Rural,  Rural  Hi-Low 


16.  ftUlUfiJLL 
Tltl»: 
Obi : 


Method ; 


Concl; 


Stren : 
Weak  : 


Abrahams,  N.M.,  Neumann,  I.,  and  Rlmland,  B. 
(April  1973) 

Prel  lmlnary  Val  ldat  1  on  of.  an  Interest  Inven¬ 
tory  for  Select  Ion  of  Navy  Recru  Iters 
To  improve  recruiter  selection  through  the 
use  of  the  Strorg  Vocational  Interest  Blank 
( 3VI 3 )  and  other  predictor  instruments. 

SVIBs  were  collected  from  samples  of  the  most 
and  least  effective  recruiters  at  36  of  42 
recruiting  3tation3.  The  responses  of  the  two 
groups  were  compared  for  one -half  of  the  sarapia 
and  used  to  establish  scoring  weights.  The 
valid  responses  were  then  assembled  into  the 
Recruiter  Interest  Scale-1  (RIS-1).  The  other 
half  of  the  3ample  were  scored  on  the  RIS-1  to 
determine  how  well  the  scale  discriminated 
between  the  most  and  least  effective  re¬ 
cruiters. 

The  SVIB  scale,  RIS-1,  proved  to  discrimin¬ 
ate  quite  well  between  the  most  and  least 
effective  recruiters.  When  scores  of  the 
"hold-out"  group  were  ordered,  the  top  quarter 
had  approximately  three  times  as  many  effective 
recruiters  as  did  the  bottom  fourth. 

Successful  in  cross-val idatlon. 
u.  Effectiveness  designations  were  made  by 
the  Commanding  Officer  -  subjective. 

b.  The  recruiters  Involved  in  the  comparison 
represented  the  extremes  in  terms  of  effective¬ 
ness  so  the  degree  of  discr iuination  achieved 
by  the  SVIB  scale  Is  greater  than  it  would  be 
In  a  sample  reprasenting  the  entire  range  of 
effectiveness . 

c.  Preliminary  findings  only,  further  research 
is  needed. 
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17. 


Title 

mi. 


Method : 

-gtren: 

WfiLilLL 


^  and  Brown  hr  .  * 
l&S.  OgygJQPment  of  A'n  Inter  1376  > 

£&&  Selection  ~f' * °r y  for 
Using  a  versio~orTh7£N?7^»rl£7bUI 
Interest  Scale)  th*  Mro?*y/!!IS  (Re«uiter 

the  authors  hoped  to  select  !f‘n<  Corpa  RIS)' 
fruiters.  s.iect  Marine  Corps  re- 

MlRIS,-  used  above-aoMr  in.» 

average  recruiters  d=;  fh  <  avef<J<le>  Jnd  below 
scaled 
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Author  : 
Title: 

Qkii 


Method : 


Concl : 


Stren : 


Weak  : 


Borman,  W.C.,  Rosse,  R.L.,  and  Touquam,  J.L. 
(Sep.  1981) 

Development  and  Val iriatlon  of  a  Recr u  Iter 
Select  ion  Battery 

To  expand  and  refine  the  test  battery  and  to 
determine  its  validity  for  predicting  various 
dimensions  of  recruiter  performance  -  third 
phase . 

The  Test  Battery  was  revised  by  including 
additional  experimental  items  selected  on  the 
basis  of  their  hypothesized  relationship  to  the 
underlying  ''constructs'*  of  the  battery.  Admin¬ 
istered  to  194  Navy  recruiters  in  seven 
different  locations.  The  two  primary  measures 
of  success  used  were  (a)  production  data 
compiled  over  a  6-month  period,  and  (b)  ratings 
gathered  from  supervisors  and  peers  on  four 
aspects  of  performance.  Factor  analysis  was 
used  to  identify  valid  constructs. 

Composites  of  the  new  items  successfully 
measured  their  target  constructs.  In  about 
half  the  cases,  validity  of  the  constructs  was 
also  improved.  The  scales  derived  from  the 
constructs  validly  predicted  recruiter 
productivity  (average  number  of  persons 
recruited)  and  rated  recruiter  performance. 

The  primary  purpose  of  this  research  was  to 
develop  measures  which  would  predict  Navy  and 
Marine  Corps  recruiter  performance.  The  pro¬ 
cedure  used  successfully  identified  personality 
and  vocational  interest  constructs  related  to 
one  or  more  aspects  of  recruiter  effectiveness, 
and  successfully  developed  additional  parallel 
measures  of  these  constructs.  This  study 
provides  additional  stability  to  the  results  of 
the  first  two  efforts  in  that  similar  solutions 
are  found  regarding  selling  skills,  human 
relations  skills,  and  organizing  skills. 

Factor  analysis  was  used  and  the  inter-rater 
reliabilities  of  the  factor  scores  completed 
were  .62,  .48,  and  .65  respectively,  which  was 
sufficiently  high  to  allow  the  factor  scores  to 
represent  individual  recruiters'  effectiveness 
in  these  three  areas  of  Navy  recruiting. 
Response  rate  data  were  not  available  for  the 
new  items  since  the  battery  administered  to  the 
applicant  sample  did  not  contain  these  items. 
(The  applicant  sample  consisted  of  131  fleet 
personnel  who  had  volunteered  for  recruiting 
duty  and  completed  the  same  inventory  battery 
that  was  administered  to  the  previous  con¬ 
current  validity  sample.) 


Pep ■  V;  Production 

T ndeo  ■ V :  Selling  skills,  Human  relations  skills, 

organizing  skill,  and  overall  performance. 

Other :  Performance  was  broken  into  two  categories, 

personality  items  and  vocational  interest 
items.  The  constructs  of  the  personality 
items  are  as  follows: 

a.  Selling  skills:  Good  impression,  impul¬ 
sive,  enjoying  being  center  of  attention, 
working  hard. 

b.  Human  relations  skills:  People  oriented, 
spontaneity  and  impulsiveness,  unhappy  and  lack 
of  confidence,  ambitious  and  working  hard. 

c.  Organizing  skills:  Order  and  planning 
ahead,  leading  and  influencing  others,  unhappy 
and  discouraged,  "bad  actor". 

”  d. Overall  performance:  Working  hard, impul- 
sive,  leading  and  influencing  others,  good 
impression,  people  oriented. 

The  constructs  of  the  vocational  interest 
items  were: 

a.  Selling  skills:  Extroverted  interests, 
interests  in  detail  work,  law  and  political 
interests,  sports  interests. 

b.  Human  relations  skills:  Extroverted 
interests,  interest  in  teaching,  "feminine" 
interests,  interest  in  newspaper  work,  sports 
interest,  religious  interests. 

c.  Organizing  skills:  interest  in  politics, 
interest  in  detail  work,  "feminine"  interests, 
leadership  interests. 

d.  Overall  performance:  Law  and  political 
interest,  extroverted  interests,  sports 
interests,  interest  in  teaching,  "feminine" 
interests . 

Ti.e  validity  of  final  keys  for  predicting 

- - production  (N=194)  were  as  follows: 

Correlation  with 


Predictor  Key  Production 

Selling  Skills  -  .22* 

Human  Relations  Skills  .23* 

Organizing  Skills  .13 

Overall  Performance  .26* 

*p  <  .01. 
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Author :  Atwater,  D.C.,  Abrahams,  N.M.,  and  Trent,  T.T. 
(May  1986) 

Title:  Va 1 Idat ion  of  the  Mat ine  Cor ds  Special  Assign¬ 

ment  Battery  ( SAB ) 

Ob j :  To  establish  objective  procedures  that  would 

improve  the  Marine  Corp's  ability  to  select  the 
most  qualified  Marines  to  recruitlnq  duty. 

Method  :  Standard  correlat i onal  techniques. 

Concl :  Recruiters  whose  scores  were  in  the  top  20\ 

obtained  27-40\  more  recruits  than  recruiters 
who  scored  in  the  lowest  20\.  Results  strongly 
confirm  findings  of  developmental  work.  The 
recruiter  selection  composite  is  related  to 
important  aspects  of  recruiter  performance. 

Pep . V . :  Product lon/SAB  Scores 

Explan :  Selling  skills;  Human  relations;  Organizing 
Skills;  Overall  performance  (664  applicants) 
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Author :  Borman,  W.  C.,  et  al.  {June  1981) 

Title:  "Recruiter  Assessment  Center:  Candidate 

Materials  and  Evaluator  Guidelines" 

Obi :  This  report  contains  materials  and  evaluation 

guidelines  for  a  series  of  exercises  to 
determine  an  individual's  potential  as  an 
Army  Recruiter.  It  outlines  each  of  six 
methods  and  then  explains  how  each  are 
evaluated.  The  six  exercises  consist  of: 

a.  Personal  Interview 

b.  Cold  Calls 

c.  Interviews  with  potential  recruits 

d.  Interviews  with  concerned  parents 

e.  Five-minute  talk  about  the  Army 

f.  In-basket  exercise 

Method :  Six  exercises  are  designed  to  simulate 

situations  in  which  recruiters  find  themselves. 
The  exercises  also  provide  opportunities  to 
assess  an  individual  on  17  personal 
character Istics . 

Concl :  This  report  does  not  offer  any  results,  only 

guidelines  to  which  the  recruiters  were 
evaluated . 


21.  Author: 
Title.; 

QhlL 


Concl : 


Borman,  W.  C.  (1982) 

"Validity  o f  Behavioral  Assessment  for 
Predicting  Military  Recruiter  Performance" 

To  determine  the  validity  of  an  assessment 
center  designed  to  select  U.  S.  Army 
recruiters . 

Assessment  Program  which  consisted  of  the 
following  phases: 

a.  Structured  interview  -  targeted  at  sub¬ 
ject's  level  of  achievement  motivation, 
potential  for  being  a  "self-starter",  and 
commitment  to  the  Army. 

b.  Cold  calls  -  phoning  three  prospects  for 
the  purpose  of  getting  them  into  the  office. 

c.  Interviews  -  Follow-up  cold  call  and 
promote  Army  enlistment  to  two  of  three  cold- 
call  prospects.  A  third  "walk-in"  was  also 
interviewed . 

(1.  Interview  with  concerned  parent  -  interview 
with  father  of  one  of  the  prospects. 

e.  Pive-minute  speech  about  the  Army  - 
delivered  to  the  rest  of  the  group  and 
assessors . 

f.  In-basket  -  subject  was  given  an  in-basket 
filled  with  notes,  phone  messages,  and  letters 
in  which  to  take  action. 

Subjects  were  57  soldiers  (all  but  one  were 
men)  entering  the  U.  S.  Army  recruiting  school. 
Assessors  were  16  experienced  and  successful 
Army  recruiters.  Validity  of  the  assessment 
ratings  was  evaluated  by  correlating  them  with 
subsequent  success  in  training. 

A  composite  of  assessment  ratings  yielded 
corrected  validities  of  almost  50\.  "First 
impression"  evaluations,  ratings  based  on 
structured  interviews,  and  test  scores 
correlated  near  zero  with  performance  in 
training.  Results  of  the  study  confirm  that 
valid  assessment  does  not  require  behavioral 
scientists  as  assessors,  and  analysis  suggest 
that  statistical  composites  of  assessment 
ratings  on  individual  exercises  may  be  slightly 
more  valid  than  "clinical"  consensus  judgements 
made  after  discussing  assessment  performance. 

Ratings  for  the  speech,  intirview  with 
parent,  in-basket,  and  interview  exercises  had 
the  highest  validities.  The  structured  inter¬ 
view  was  less  valid  than  simulation  exercises, 
and  tests  were  also  low  in  validity  as  well. 
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Stren :  The  research  design  provided  an  opportunity  to 

evaluate  the  following  for  success  in  training: 

a.  First  impression,  physical  attractiveness, 
and  likability. 

b.  Structured  interview. 

c.  Test  scores  on  a  personality  Inventory 
previously  developed  to  select  recruiters. 

d.  Subject's  performance  in  individual 
exercises . 

e.  Consensus  ratings  of  subjects  reached  after 
assessors  discussed  their  performance  each  day. 

Also,  there  was  virtually  no  chance  of 
criterion  contamination  because  the  assessment 
ratings  were  completely  independent  of 
subject'3  performance  during  training. 

Weak :  a.  Small  sample  size. 

b.  Costly. 

c.  The  range  of  assessment  ratings  was  re¬ 
stricted  since  seven  people  who  were  evaluated 
during  the  assessment  program,  but  not  included 
in  the  validity  analysis  because  they  dropped 
out  before  training  tended  to  either  have  very 
high  or  very  low  assessment  ratings.  Also, 
four  more  subjects  with  low  assessment  ratings 
were  eliminated  because  of  inadequate 
performance . 
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Author : 


Title: 


Method : 


Concl : 


Stren : 


Weak  : 


Pep .  V: 
Explan : 


weltin,  M.M.,  Fr  1  email,  s.,  Ellg,  T.,  and 
Johnson,  R.M.  (Nov.  1985) 

"Predicting  Army  Recruiters'  Job  Performance 
from  Development  Center  Ratings" 

To  relate  the  ratings  of  the  original  sample 
and  a  subsequent  'development  center'  sample  to 
a  measure  of  job  productivity,  the  number  of 
contracts  the  new  recruiter  produced  in  his/her 
first  year  on  the  job. 

Assessors  rated  57  potential  recruiters  on 
eleven  personal  character istics,  such  as 
persuasiveness,  sociability,  flexibility,  and 
practical  judgement,  as  displayed  in  several 
different  situational  execises.  Also  used 
stepwise  regression. 

Assessment  center  ratings  had  low  correlations 
with  job  performance  in  a  small  sample  and  in 
the  large  sample,  correlations  were  significant 
for  a  combined  cold  /call  interview  exercise 
and  speech  exercise  with  job  performance. 
Productivity  of  the  recruiter's  battalion  was 
the  single  most  important  factor  in  predicting 
his/her  job  performance.  Speech  and  AFQT 
predicted  approximately  2%  additional  variance. 

a.  The  recruiter  development  center  should  be 
upgraded  to  the  quality  of  the  original 
assessment  center. 

b.  Flexibility  and  sociability  are  not 
adequate  rating  dimensions. 

c.  Sales  traininq  technologies  would  be  most 
beneficial  to  improving  the  recruiter's 
effectiveness  and  motivation. 

The  assessment  center  ratings  showed  some 
utility  as  predictors  of  recruiters  job 
per  forma nee . 

a.  Although  the  assessment  center  sample  had 
good  quality  rating,  sample  size  was  too  small 
to  generalize  results. 

b.  Development  center  ratings  and  training 
school  grades  were  compromised  by  changes  that 
took  place  in  the  operationalization  of  the 
assessment  center 

Job  Performance 

Ratings:  Cold  call,  interview,  speech, 
inbasket,  composite;  Training  Grades:  Written 
test.  Phase  II  (performance),  interviewing, 
telephoning;  Other  predictors:  Navy  test,  ARI 
test,  GPP  (sociability),  GPI  (cautiousness). 
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23.  Author :  Bennett,  J.T.  and  Haber,  S.E.  (June  1973) 

Title:  Select  ion.  Deployment  and  Evaluation  of  Marine 

Recruiters 

Ob j :  To  determine  the  factors  that  influence  Marine 

Corps  recruiters 
Method :  Multiple  regression 

Cone  1 :  a.  Urban  and  suburban  recruiters  enlisted  more 

people  per  month  than  rural  recruiters. 

b.  Recruiters  in  their  home  state  enlisted 
more  than  those  stationed  more  than  500 
miles  from  home  state. 

(a  &  b  are  results  from  high  enlistment  area) 

c.  Those  who  felt  recruiting  duty  was  a 
financial  hardship  enlisted  more  people 
per  month  than  those  who  did  not. 

d.  Recruiters  with  prior  service  as  career 
planners  were  more  productive  than  those 
who  did  not. 

(c  &  d  are  results  from  low  enlistment  area) 
Regression  equations  were  not  cross-validated. 


Weak : 


24 .  Author  : 
Title: 

Obi: 

Method : 


Concl : 


Str en : 


Weak  : 


Dep.  V.  : 
Ind .  V.  : 


Best,  J.B.  and  Wylie,  W.J.  (June  1974) 

Using  Navy  Recruiter  Attributes  and  Attitudes : 

A  Survey  Analysis 

To  predict  recruiter  performance  using 
recruiter  characteristics. 

Survey  Analysis,  Multiple  regression  analysis. 
Data  was  collected  from  a  sample  of  49  active 
U.  S.  Navy  recruiters  assigned  within  the  San 
Francisco  Recruiting  District.  Used  survey 
interviews  to  identify  attributes  of  effective 
recruiters.  After  interviews,  additional  bio¬ 
graphical,  career,  and  educational  data  were 
obtained  from  the  service  records  of  recruiters 
who  were  interviewed. 

The  most  favorable  aspect  of  recruiting  duty 
was  independent  duty.  Least  desired  was  public 
speaking.  Over  one-third  of  the  recruiters 
felt  their  particular  stations  were  overmanned 
while  an  equal  number  considered  the  station 
work  goals  to  be  too  high.  One-fourth  of  the 
recruiters  were  overweight. 

a.  Attempt  was  made  to  equalize  percentage 
representation  in  the  sample  of  each  of  six 
zones  within  the  district. 

b.  Clustered  sampling  techniques  were  used  to 
decrease  t lie  physical  variables  into  such 
groupings  as  geographic  differences  (coastal 
vs.  inland),  community  type  (urban,  suburban, 
rural),  and  station  size. 

c.  Identified  subjective  attributes  based  on 
survey  interviews  which  could  be  beneficial  to 
expert  systems  model. 

a.  Failed  in  cross-validation. 

b.  Looked  at  only  one  recruiting  district,  not 
necessarily  representative. 

c.  A  control  group  data  set  was  not  available 
for  comparative  purposes. 

d.  Used  Commanding  Officer's  evaluations  of 
each  individual's  "effectiveness"  on  a  scale  of 
1  to  5  (5  being  the  most  effective). 

Commanding  Officer  used  his  own  definition  of 
effectiveness . 

Command  evaluation  of  each  recruiter  (CRUTVAL). 

a.  The  area  the  recruiter  was  from  in  terms  of 
urban,  suburban,  rural  (HOMAREA). 

b.  Age  (AGE). 

c  .  GCT  score  ( GCT ) . 

d.  Total  years  of  active  military  service 
( YRSSVC ) . 

e.  Proximity  of  HOMAREA  to  a  major  body  of 
water  by  distance:  (1)  Less  than  20  miles; 

(2)  20  to  200  miles;  (3)  Greater  than  200 
miles  (HOMPROX). 
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APPENDIX  B 


THE  EXPERT  SYSTEMS: 

INDICES ,  VARIANCE,  AND  MEAN  SQUARED  ERROR 

Expert  #1. 


Fidelity 

Index 

Standards 

Index 

Discrim. 

I  ndex 

Var iance 
ExDla i ned 

Overall 

Profile 

30.3 

65.5 

18.1 

81.54 

Comm. 

Skills 

88.2 

66.5 

41.8 

77.86 

Demog. 

Charac. 

97.8 

72.7 

23.7 

95.82 

Military 

Backg. 

89.6 

72.7 

22.6 

80.37 

Person. 

Charac. 

87.1 

74.0 

25.2 

75.96 

Behavior 

Charac. 

81.7 

73.0 

20.9 

66.79 

Specific 

Exp. 

77.8 

70.3 

13.9 

60.65 

Mean- 
Sg . Err 

1.95 

4.92 

1.21 

2.50 

3.10 

3.01 

2.18 
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The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Expert  *2. 


Fidelity 

Index 

Standards 

I  ndex 

Discrim. 

Index 

Var lance 
Exda  ined 

Overall 

Profile 

36.1 

80.0 

78.6 

74.18 

Com. 

Skills 

88.2 

56.0 

96.0 

77.89 

Denog. 
Char ac . 

97.5 

50.0 

79.4 

95.16 

Military 

Backg. 

97.7 

34.4 

111.9 

95.59 

Person. 

Charac. 

92.0 

78.3 

99.8 

84.71 

Behavior 

Charac. 

92.0 

67.5 

102.7 

84.65 

Specific 

Exp. 

96.1 

33.8 

70.4 

92.39 

Mean- 
So . Er r 

9.98 

11.67 

4.37 

5.87 

9.75 

10.06 

4.85 
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Expert  #3. 


The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Fidelity 

Index 

Standards 

Index 

Discrim. 

Index 

Var iance 
Explained 

Mean- 
Sa . Err 

Overall 

Profile 

89.4 

99.3 

55.9 

80.03 

6.25 

Coma. 

Skills 

94.5 

43.5 

82.0 

89.43 

6.66 

Demog. 
Charac . 

95.0 

49.2 

92.1 

90.40 

7.13 

Military 

Backg. 

91.1 

59.4 

31.2 

83.08 

3.21 

Person. 

Charac. 

95.4 

67.0 

85.2 

91.18 

6.33 

Behavior' 

Charac. 

93.9 

65.0 

53.7 

88.18 

4.61 

Specific 

Exp. 

89.4 

56.3 

17.3 

80.00 

1.94 
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The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Expert  M . 

Fidel ity 


Index 

Overall 

Profile 

92.8 

Com. 

Skills 

90.1 

Deaog. 

Charac. 

75.4 

Military 

Backg. 

87. C 

Person. 

Charac. 

85.7 

Behavior 

Charac. 

89.5 

Specific 

Bxp. 

96.6 

Standard  -i 
Index 

Discr  i; 
Index 

59.2 

79.2 

49.0 

88.0 

48.6 

70.3 

28.1 

69.1 

80.0 

86.9 

59.0 

50.4 

37.8 

61.8 

Var iance 
Explained 

Mean- 

SSLt&LL 

86.19 

7.36 

81.35 

9.50 

56.86 

11.54 

75.81 

8.50 

73.60 

11.16 

80.20 

5.61 

93.48 

3.95 

The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Expert  #5. 

Fide  1 i ty 
Index 


Overall 

Profile 

91.5 

Com. 

Skills 

93.1 

Deaog. 

Charac. 

93.6 

Military^ 

Backg. 

96.7 

Person. 

Charac. 

91.3 

Behavior 

Charac. 

96.9 

Specific 

Bxp. 

96.5 

Standards 

Index 

Discr im 
Index 

80.0 

17.7 

66 .  C 

73.2 

50.0 

70.7 

50.0 

66.3 

79.0 

75.1 

61.0 

60.5 

52.3 

56.2 

Variance  Mean- 
Explained  So. Err 


89.39 

3.88 

87.26 

6.53 

87.70 

6.20 

93.61 

1.18 

88.97 

6.26 

93.95 

3.72 

93.19 

3.67 
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The  Expert  Systems: 

Indices,  Variance,  and  Mean*Squared  Error 

Expert  16 . 


Fidelity 

I  ndex 

Standards 

Index 

Discr im. 

I  ndex 

Variance 

ExDlained 

Mean- 
3a . Err 

Overall 

Profile 

97.9 

58.0 

62.2 

96.02 

3.10 

Com. 

Skills 

96.0 

42.4 

76.5 

92.17 

5.35 

Deaog. 

Charac. 

96.4 

51.6 

64.4 

93.00 

4.26 

Military 

Back?. 

89.2 

40.6 

61.4 

79.74 

6.92 

Person. 

Charac. 

95.6 

57.0 

58.5 

91.50 

4.27 

Behavior 

Charac. 

96.9 

60.5 

55.6 

93.91 

3.43 

Specific 

Exp. 

93.5 

29.7 

62.4 

87.51 

5.51 
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The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 

Expert  #7. 


Fidelity 

Standards 

Discr im. 

Variance 

Mean- 

I  ndex 

Index 

Index 

ExDla ined 

Sa . Er r 

Overall 

Profile 

95.9 

51.4 

43.7 

92.08 

3.08 

Coon. 

Skills 

87.5 

42.6 

68.8 

76.73 

8.29 

Demog . 
Charac. 

93.6 

45.0 

59.5 

87.77 

5.20 

Military 

Backg. 

86.6 

35.6 

55.9 

75.04 

6.99 

Person. 

Charac. 

90.8 

56.9 

57.6 

82.48 

6.03 

Behavior 

Charac . 

92.8 

55.0 

39.7 

86.24 

3.68 

Specific 

Exp. 

93.4 

48.9 

23.7 

87.39 

2.10 

Indices, 

Expert  1 8 . 


The  Expert  Systems: 

Variance,  and  Mean-Squared  Error 


Fidelity  Standards 


Index 

I  ndex 

Overall 

Profile 

91.2 

77.0 

Com. 

Skills 

87.7 

45.0 

Deaog. 

Charac. 

93.6 

42.2 

Military 

Back?. 

91.5 

35.9 

Person. 

Charac. 

78.3 

67.0 

Behavior 

Charac. 

82.4 

81.6 

Specific 

Exp. 

92.2 

67.2 

Oiscr im. 
Index 

Var iance 
Explained 

Mean- 

SslJLli 

83.8 

83.27 

8.57 

74.2 

76.93 

8.91 

57.8 

87.63 

5.09 

52.9 

83.85 

5.31 

81.4 

61.33 

12.65 

92.4 

67.90 

13.09 

56.1 

85.01 

5.43 
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The  Expert  System*: 

Indices,  Variance,  and  Mean-squared  Error 


Expert  »9. 

Fidelity 


IM?x 

Overall 

Profile 

83.3 

Com. 

Skills 

88.0 

Daaoq. 

Charac. 

78.1 

Military 

Back?. 

90.4 

Parson. 

Charac. 

75.4 

Behavior 

Charac. 

83.3 

Specific 

Exp. 

92.7 

Standards 

Index 

Discr  ini 
Index 

62.0 

69.9 

58.0 

86.0 

53.1 

102.8 

35.9 

94.8 

63.0 

92.7 

71.1 

93.4 

29.8 

84.9 

Variance 

ExDlained 

Mean- 
Sc .Err 

69.43 

9.66 

77.52 

10.19 

61.04 

16.05 

81.86 

10.10 

56.90 

15.21 

69.47 

12.10 

86.07 

7.92 

The  Expert  Systems: 

Indices,  Var lance,  and  Mean~Squared  Error 


Expert  *10. 


Fidelity 

Index 

Overall 

Profile 

95.4 

Com. 

Skills 

94.6 

Deaog. 

Charac. 

96.5 

Military 

Backq. 

91.0 

Parson. 

Charac. 

94.1 

Behavior 

Charac. 

96.0 

Specific 

Exp. 

96.5 

Standards  Discrim 


IM££ 

IP0B* 

61.0 

89.1 

64.0 

79.9 

54.7 

66.7 

52.3 

68.6 

79.1 

100.8 

66.5 

84.4 

48.4 

71.0 

Variance  Mean- 


91.13 

6.63 

89.49 

6.47 

93.16 

4.36 

82.96 

7.08 

88.61 

8.50 

92.22 

5.89 

93.28 

4.60 

The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Expert  #11. 


Fidelity 

Inflqx 

Standards 

Index 

Discrim. 

Index 

Variance 

Explained 

Mean- 
Sa . Err 

Overall 

Profile 

86.7 

70.0 

76.0 

75.34 

9.44 

Comm. 

Skills 

81.7 

38.3 

124.2 

66.88 

17.87 

Deatog . 
Charac . 

95.8 

32.8 

74.5 

91.82 

5.33 

Military 

Backg. 

96 . 3 

48 . 4 

78.1 

92.76 

5.25 

Person. 
Charac . 

85.0 

87.1 

95.0 

72.40 

12.47 

Behavior 

Charac. 

95.1 

63.0 

79.3 

90.57 

6.09 

Specific 

Exp. 

97.0 

31.3 

85.4 

94.25 

5.12 

The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 


Expert  *12. 


Fidelity 

I  ndex 

Standards 

Index 

Overall 

Profile 

95.0 

17.8 

Com. 

Skills 

96.6 

6.9 

Deaog. 

Charac. 

92.7 

5.5 

Military 

Backg. 

98.6 

28.0 

Person. 

Charac. 

98.0 

30.9 

Behavior 

Charac. 

96.0 

20.1 

Specific 

Exp. 

9  4.3 

11.9 

Discr im. 

I  ndex 

Var iance 
Exda  ined 

Mean- 

Sa.Err 

103.8 

90.38 

8.05 

90.3 

93.37 

5.81 

58.9 

86.00 

5.51 

70.5 

97.32 

2.89 

121.9 

96.13 

6.00 

101.4 

92.34 

7.02 

74.3 

88.99 

6.16 
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...  The  Expert  Systems: 

indices/  Variance/  and  Mean-Squared  Error 

Expert  #13. 


Fidelity 

Index 

Standards 

Index 

Discrim. 

Index 

Variance 

ExDlained 

Mean- 
Sa .Err 

Overall 

Profile 

96.0 

55.5 

62.2 

92.16 

4.36 

Coon. 

Skills 

96.7 

64.0 

62.1 

93.59 

3.93 

Demog. 

Charac. 

98.2 

62.5 

46.9 

96.48 

2.20 

Military 

Backg. 

89.4 

57.8 

46.3 

80.00 

5.18 

Person. 

Charac. 

97.3 

73.5 

66.0 

94.83 

3.75 

Behavior 

Charac. 

95.9 

68.0 

62.0 

92.11 

4.41 

Specific 

Exp. 

95.2 

49.2 

53.1 

90.78 

4.03 
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Indices, 

The  Expert 

Systems : 

Variance,  and 

Mean-Squared  Error 

Expert  #14. 

Fidel 1 ty 
Index 

Standards 

I  ndex 

Discrim. 

I  ndex 

Var iance 
Explained 

Mean- 
Sq . Er r 

Overall 

Profile 

88.7 

84.7 

82.3 

78.82 

9.50 

Com. 

Skills 

85.9 

92.7 

64.5 

73.85 

8.39 

Demog. 

Charac. 

96.0 

82.0 

114.8 

92.29 

8.02 

Military 

Backg. 

69.1 

41.6 

49.5 

47.83 

8.94 

Person. 

Charac. 

85.6 

100.3 

85.0 

73.41 

11.16 

Behavior 

Charac. 

79.6 

109.7 

67.2 

63.45 

10.20 

Specific 

Bxp. 

86.2 

31.3 

62.4 

74.36 

7.91 

The  ExPert  Systems: 

indices.  Variance,  and  Mean-Squared  Error 


Expert  #15. 


Fidelity 

Index 

Standards 

Index 

Overall 

Profile 

94.1 

62.0 

Co  am. 
Skills 

90.7 

60.6 

Deraog. 
Charac . 

93.4 

76.6 

Military 

Backg. 

97.5 

69.5 

Person. 
Charac . 

93.0 

69.3 

Behavior 

Charac. 

92.9 

58.5 

Specific 

Exp. 

93.4 

68.8 

Discrim. 

I  ndex 

Variance 
ExDla i ned 

Mean- 
Sa . Err 

88.1 

88.62 

7.95 

80.7 

82.32 

8.48 

59.5 

87.30 

5.29 

83.7 

95.14 

5.28 

78.7 

86.66 

7.25 

61.1 

86.44 

5.63 

67.8 

87.28 

6.05 

132 


T  The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 

Expert  #16. 


Fidelity 

I  ndex 

Standards 

I  ndex 

Discrim. 

Index 

Variance 

Explained 

Kean- 
Sq . Err 

Overi.ll 

Prefile 

92.6 

60.7 

92.4 

85.83 

8.65 

Com  i. 

3k  11» 

50.2 

71.8 

76.3 

81.  J8 

8.21 

r  i«og. 

herac. 

97.1 

59.7 

100.2 

94.31 

6.08 

Military 

Backg. 

92.4 

78.1 

64.3 

85.44 

6.14 

Person. 

Charac. 

93.5 

47.4 

98.8 

87.59 

8.70 

Behavior 

Charac. 

92.3 

57.4 

84.3 

85.33 

7.89 

Specific 

Exp. 

95.1 

39.8 

74.1 

90.47 

5.72 

_  Th«  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 

Expert  MftGR 


Fide  1 i ty 
Index 

Standards 

Index 

Discrim. 

Index 

Var iance 
SxDla ined 

Mean- 

Sa . Err 

Overall 

Profile 

97.3 

73.5 

40.9 

94.78 

2.73 

Comm. 

Skills 

98.2 

53.5 

64.8 

96.55 

2.82 

Demo g . 
Charac. 

96.6 

53.4 

58.3 

93.47 

3.73 

Military 

Backg. 

96.7 

46.4 

45.7 

93.61 

2.89 

Person. 

Charac. 

96.2 

74.0 

60.9 

92.62 

4.14 

Behavior 

Charac. 

97.5 

46.3 

48.4 

95.21 

2.65 

Specific 

Exp. 

96.7 

46 . 3 

38.3 

93.70 

2.40 
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The  Expert  Systems: 

Indices,  Variance,  and  Mean-Squared  Error 
Expert  MRA 


Fidel ity 
Index 

Standards 

I  ndex 

Discr im. 
Index 

Variance 

Explained 

Mean- 
Sa .Err 

Overall 

Profile 

37.0 

59.9 

62.1 

94.17 

3.73 

Comm. 

Skills 

96.5 

54.8 

68.1 

93.21 

4.31 

Demog. 
Chirac . 

38.0 

51.4 

60.6 

96.07 

3.01 

Military 

Backg. 

96.9 

48.3 

53.0 

94.01 

3.24 

Person. 

Charac. 

97.1 

67.5 

66.8 

94.35 

3.94 

Behavior 

Charac. 

96.4 

65.2 

53.4 

92.97 

3.35 

Specific 

Exp. 

97.9 

42.7 

51.8 

95.97 

2.60 
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Communicjt 1  on  Skills 
30 .49  * 


Public  Speaking 
Skills 

Writing  Skills 
Listening  Skills 

Informing - — 

Persuad i ng 


2i-,19 

25.03 


ILii 


Demographic  Characteristics 


32.09 

Ag*  19.17 

Family  Support  33,32 

AFQT  37.01 

- College  Experience  10  .  qq 


Military  Background 
6.33 


Personality  CharacfcorUMnc 
197oT  - - 


Paygrade 

23.72 

Self-image 

31.38 

Years  of  Svc. 

(Act.) 

23.72 

Integrity 

29.43 

Years  of  Svc. 

Extroverted 

17.27 

( Res . ) 

52.55 

Sen3e  of  Humor 

16.36 

People-Oriented 

5.56 

Behavior  Character i st i c« 

. . . •  7  D - 

6.53 

Self -starter 

19.5  0 

Sales  Experience 

45.18 

Commitment 

17.45 

Public  Speaking  Exp. 

30.90 

Flexibility 

5.13 

Counselling  Exp. 

23.92 

Attention  to 

Detail 

43.33 

Decisiveness 

14 . 58 
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Profile  of  the  Suecessf ul  Recruiter 


Expert  »2. 


Communication  Skills 
29.90 


Demographic  Character  1st lcs 
17.63 


PuPl Ic  Speak ing 
Skills 

8.34 

Age 

5.24 

Family  Support 

50 .39 

Writing  Skills 

2.84 

AFQT 

33.22 

Listening  Skills 

3  5.34 

College  Experience 

5.65 

Informing 

25.84 

Persuading 

27.64 

Military  Background 

Personality  Character ist lcs 

4.52 

12.32 

Paygrade 

84 . 57 

Self-Image 

17.95 

Years  of  Svc. 

Integrity 

60.84 

(Act. ) 

5.36 

Extroverted 

2.90 

Years  of  Svc. 

(Res  .  ) 

9  .  57 

Sense  of  Humor 

10.66 

People-Or iented 

7.65 

Behavior  Character ist ics 

Specific  Experience 

12.27 

22.87 

Seif  -Starter 

21 .82 

Sales  Experience 

37.65 

Commi tment 

40.21 

Public  Speaking  Exp. 

5.  44 

Flexibility 

16 . 49 

Counselling  Exp. 

56.91 

Attention  to 

Detail 

11.43 

Dec  is iveness 

10.06 
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Profile  o f  the 

Expert  #3. 

Communieat ion  Skills 


24 .52 

Public  Speaking 
Skills  3.93 

Writing  Skills  4  ■  09 

Listening  Skills  23.58 

Informing  26.74 

Persuading  35.66 


Success f ul  Recru i ter 


Demographic  Character ist ics 


19 . 09 

Age  2 . 70 
Family  Support  27.32 
AFQT  62.61 
College  Experience  7.36 


Mi  1 i tar y  Background 
6.71 


Paygrade 

33. 

.  33 

Years  of 

Svc . 

(Act. ) 

33. 

.33 

Years  of 

Svc . 

(Res.  ) 

33. 

.33 

Behavior  Character ist ics 

12.69 

Self -Starter 

32.82 

Commi tment 

12.27 

Flexibility 

20.76 

Attention  to 
Detail 

30.01 

Decisiveness 

4.14 

Personal lty  Character  1st ics 


18.75 

Self-Image  24 . 16 
Integrity  44.91 
Extroverted  14.20 
Sense  of  Humor  5.43 
People-Or  ierited  11.30 


Sped  f  ic  Exper  ience 
18.23 

Sales  Experience  33.33 
Public  Speaking  Exp.  33.33 
Counselling  Exp.  33.33 
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Profile  of  the  Successful  Recruiter 


Expert  H. 

Communlcat ion  Skills 


26.68 

Public  Speaking 
Skills 

10.26 

Writing  Skills 

8 , 78 

Listening  Skills 

13.21 

Informing 

21.08 

Persuading 

46.67 

Military  Background 
51.73 

Paygrade 

38.70 

Years  of  Svc. 
(Act. ) 

14.08 

Years  of  Svc. 
(Res. ) 

47.22 

Behavior  Character istics 

8.62 

Self-Starter 

30 . 67 

Commitment 

13.89 

Flexibility 

28.54 

Attention  to 
Detail 

15.41 

Dec  is i veness 

11.48 

Demographic  Characteristics 


5.18 

Age  13,08 
Family  Support  22.69 
AFQT  38 ■ 33 
College  Experience  25.90 


Persona  1 1  tv  Characteristics 


2.30 

Self-Image  14.48 
Integrity  43.22 
Extroverted  6.07 
Sense  of  Humor  16.19 
People-Oriented  20.03 


Specif ic  Experience 
5.49 

Sales  Experience  58.17 
Public  Speaking  Exp.  20.99 
Counselling  Exp.  20.84 
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Profile  of  the 

Expert  #5. 

Communication  Skills 


14.01 

Public  Speaking 
Skills  17.32 

Writing  Skills  3.12 

Listening  Skills  44,44 

Informing  25.06 

Persuading  7.57 


Successful  Recruiter 


Demographic  Character istic3 


33.78 

Age  4 . 75 
Family  Support  33.33 
AFQT  38 . 08 
College  Experience  23.83 


Military  Background 


14.71 


Paygrade 

46.63 

Years  of 
(Act. ) 

Svc . 

23.96 

Years  of 
( Res . ) 

Svc . 

29.41 

Behavior  Character ist lcs 


9.57 

Self-Starter 

25.09 

Commi tment 

23.92 

Flexibility 

8.52 

Attention  to 
Detail 

18.04 

Decisiveness 

24.43 

Personality  Character istics 

18.76 

Self-Image 

14.01 

Integrity 

50.69 

Extroverted 

16.43 

Sense  of  Humor 

1.99 

People -Or iented 

16 . 88 

Specific  Experience 
9.18 

Sales  Exper  ience  14.41 
Public  Speaking  Exp.  35.01 
Counselling  Exp.  50.58 
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Profile  of  the  Success! ul  Recru 1  ter 


Expect  »6. 


Communication  Skills 

Demographic  Character ist ics 

27.69 

9.62 

Public  Speaking 

Age 

10.99 

Skills 

1.63 

Family  Support 

34.70 

Writing  Skills 

8.33 

AFQT 

44 . 51 

Listening  Skills 

32.37 

College  Experience 

9.80 

Informing 

17.96 

Persuading 

39.71 

Mi lltary  Background  Personal ity  Character ist Lcs 


7.60 

26.90 

Paygrade 

56 .07 

Self-Image 

17.42 

Years  of  Svc. 

Integrity 

35.81 

(Act. ) 

26.09 

Extroverted 

9.40 

Year3  of  Svc. 

(Res . ) 

17.84 

Sense  of  Humor 

10.25 

People-Oriented 

27.13 

Behavior  Character  1st ics 

Specific  Experience 

22.79 

5.40 

Self-Starter 

28.39 

Sales  Experience 

42.18 

Commitment 

16 . 34 

Public  Speaking  Exp. 

28.45 

Flexibility 

28 .85 

Counselling  Exp. 

29 . 38 

Attention  to 

Deta i 1 

9 .28 

Decisiveness 

17.13 
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Expert  MAGR 

Communication  Skills 
27.71 

Public  Speaking 
Skills  8.16 

Writing  Skill3  4 .14 

Listening  Skills  30.50 

Informing  25.90 

Persuading  31.29 


Behavior  Character ist ics 


EBB 


Self-Starter  30.06 
Commitment  27.01 
Flexibility  23,01 


Attention  to 
Detail 


12.04 


sful  Recruiter 


Profile  of  the  Success f ul  Recr  u Iter 


Expert  V 7 . 

Communciat ion  Skills 
28.28 


Public  Speaking 
Skills 

3  8,34 

Writing  Skills 

10.57 

Listening  Skills 

23.66 

Informing 

14 .12 

Persuading 

33. 3L 

Military  Background 

13.70 

Paygrade 

44.32 

Years  of  Svc. 
(Act. ) 

32.15 

Years  of  Svc. 
(Res. ) 

23.52 

Behavior  Character ist ics 

17.12 

Self -Starter 

27.94 

Commitment 

19.23 

Flexibi 1 ity 

15.98 

Attention  to 
Detail 

24.94 

Decisiveness 

11.90 

Demographic  Characteristics 


9.77 

Age  3 . 78 
Family  Support  33.32 
AFQT  43.87 
College  Experience  18.54 


Personal Ity  Characteristics 


24.99 

Self-Image  29 . 50 
Integrity  41.66 
Extroverted  17 , 53 
Sense  of  Humor  4.46 
People-Oriented  6.85 


Spec  1 f ic  Exper lence 
6  .13 

Sales  Experience  20.04 
Public  Speaking  Exp.  15.27 
Counselling  Exp.  64.69 
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Profile  of  the  Successf ul  Recruiter 


Expert  *8. 

Conununlcatlon  Skills 


43.65 

Public  Speaking 
Skills  20.13 

Writing  Skills  10.45 

Listening  Skills  29.27 

Informing  31.91 

Persuading  8.25 


Ml  1 i tarv  Background 
3.53 

Paygrade  20.33 

Years  of  Svc. 

(Act.)  45.58 

Years  of  Svc. 

(Res.)  34.10 

Behavior  Character  1st ics 
10.60 


Self-Starter  9.44 

Commitment  35.56 

Flexibility  37.95 

Attention  to 

Detail  11.02 

Decisiveness  6.03 


Demographic  Character istics 


4.75 

Age  32.56 
Family  Support  7.78 
AFQT  41.38 
College  Experience  18.29 


Personality  Character  1st ics 


25.65 

Self-Image  6 . 58 
Integrity  35.18 
Extroverted  12.21 
Sense  of  Humor  11.59 
People-Oriented  34.45 


Spec  1 f ic  Experience 
11.81 

Sales  Experience  22.27 
Public  Speaking  Exp.  45.32 
Counselling  Exp.  32.41 
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Profile  of  the  Successful  Recruiter 


Expert  19 . 

Communication  Skills  Demographic  Character lstlcs 

11.11  17.16 


Public  Speaking 
Skills 

5.56 

Age 

LLJ2 

Family  Support 

1^11 

Writing  Skills 

9.62 

AFQT 

57.94 

Listening  Skills 

27.17 

College  Experience 

2  3  .--7  3 

Informing 

11.02 

Persuading 

46.63 

Military  Background 

Personality  Characteristics 

15.30 

29.62 

Paygrade 

62.44 

Self-Image 

15.52 

Years  of  Svc. 

Integrity 

26.41 

(Act. ) 

29.73 

Extroverted 

33.88 

Years  of  Svc. 

(Res. ) 

7.83 

Sense  of  Humor 

20.96 

People-Oriented 

3.23 

Behavior  Character ist ics 

Specific  Experience 

15.20 

11.60 

Self-Starter 

34.45 

Sales  Experience 

46.95 

Commitment 

32.48 

Public  Speaking  Exp. 

32.54 

Flexibility 

3.36 

Counselling  Exp. 

20.50 

Attention  to 

Detail  15.64 


Decisiveness  14.08 
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Profile  of  the  Successful  Recruiter 


Expert  #10. 


Communication  Skills 

Demographic  Character istics 

54.61 

5.41 

Public  Speaking 

Age 

8.84 

Skills 

15.66 

Family  Support 

26.04 

Writing  Skills 

6.11 

AFQT 

45.58 

Listening  Skills 

13.82 

College  Experience 

19.54 

Informing 

27.42 

Persuading 

36.99 

Military  Background 

Personality  Character istics 

10.50 

5.96 

"Paygrade 

37.81 

Sel f-Image 

14.50 

Years  of  Svc. 

Integrity 

53.68 

(Act. ) 

56.61 

Extroverted 

10.54 

Years  of  Svc. 

{ Res . ) 

5.58 

Sense  of  Humor 

14.27 

People -Or iented 

7.02 

Behavior  Character ist ics 

Specific  Experience 

21 .44 

2.08 

Self -Starter 

3.76 

Sales  Experience 

26.06 

Commitment 

9.70 

Public  Speaking  Exp. 

44.78 

Flexibility 

11.51 

Counselling  Exp. 

29 . 16 

Attention  to 

Detail 

56 .86 

• 

Decisiveness 

18.18 
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Profile  of  the  Successful  Recruiter 


Expert  #11. 
Communication  Skills 


29.95 

Public  Speaking 
Skills  9.24 

Writing  Skills  12 . 36 

Listening  Skills  31.77 

Informing  21.06 

Persuading  25.06 


Mill tarv  Backgr ound 
18.07 

Paygrade  62.46 

Years  of  Svc . 

(Act.)  21 .24 

Years  of  Svc. 

(Res.)  16.29 

Behavior  Character ist lcs 


30.53 

Self-Starter  35.32 

Commitment  14.26 

Flexibility  16.68 

Attention  to 

Detail  10.53 

Decisiveness  23.21 


Demographic  Character  1st  lcs 


12.02 

Age  21.09 
Family  Support  35.97 
AFQT  32.12 
College  Experience  10 . 82 


Personal ity  Characteristics 


15.47 

Self-Image  16 ■ 88 
Integrity  34.75 
Extroverted  27 . 39 
Sense  of  Humor  9. 30 
People-Oriented  11 . 68 


Spec i f 1c  Experience 
1.95 

Sales  Experience  31 . 25 
Public  Speaking  Exp.  31.25 
Counselling  Exp.  37 . 50 
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Expert  *12 


Profile  of  the  Successful  Recruiter 


Communication  Skills 
44.76 

Public  Speaking 
Skills  23 . 50 


Writing  Skills 

11.29 

Listening  Skills 

12.35 

Informing 

7.92 

Persuading 

44.95 

Military  Background 

9.18 

Paygrade 

11.40 

Years  of  Svc. 
(Act. ) 

81.05 

Years  of  Svc. 
(Res. ) 

7.55 

Demographic  Character ist ics 


8.74 

Age  14.69 
Family  Support  36.13 
AFQT  16.00 
College  Experience  33.18 


Personality  Characteristics 
8.04 

Self-Image 

1.88 

Integrity 

81.21 

Extroverted 

8.84 

Sense  of  Humor 

1.00 

People -Or iented 

7.07 

Behavior  Character ist ics 
11.82 


Spec i f ic  Experience 
17.45 


Self -Star  ter 

54.12 

Sales  Experience 

71 . 47 

Commitment 

12.35 

Public  Speaking  Exp. 

19.24 

Flexibility 

9.12 

Counselling  Exp. 

9 . 29 

Attention  to 
Detail 

20.95 

Decisiveness 

3.46 
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Prof i le  of  the  Successful  Recruiter 


Expert  #13. 
Communication  Skills 


29.68 

Public  Speaking 
Skills  12, 

Writing  Skills  3.30 

Listening  Skills  22.47 

Informing  17,78 

Persuading  44.02 


Mi  1 i tar v  Backgr  ound 
10.17 

Paygrade  42 . 21 

Years  of  Svc. 

( Act . )  40.88 

Years  of  Svc. 

(Res.)  16.92 

Behavior  Character ist lc3 


24 . 53 

Self-Starter  12.45 

Commitment  50.38 

Flexibility  13.61 

Attention  to 

Detail  10.52 

Decisiveness  13.03 


Demographic  Character  1 st ics 

0.68 

Age 

llxll 

Family  Support 

49.23 

AFQT 

30.05 

College  Experience 

8.97 

Personality  Character istlcs 


27.47 

Self-Image  5 . 66 
Integrity  50 .49 
Extroverted  27. 26 
Sense  of  Humor  3 . 19 
People-Oriented  13.41 


Specific  Experience 
7.46 

Sales  Experience  34.91 
Public  Speaking  Exp.  23.54 
Counselling  Exp.  41.55 
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Expert  #14 


Profile  o f  the 


Communicat ion  Skills 


38.11 

Public  Speaking 
Skills  8.45 

Writing  Skills  18.44 

Listening  Skills  33.18 

Informing  11,11 

Persuading  28.82 


Military  Backqr ound 

8.88 


Paygrade 

56.15 

Years  of 
(Act. } 

Svc . 

37.35 

Years  of 
(Res.  ) 

Svc. 

6.49 

Behavior  Character ist ics 


20.81 

Self-Starter 

25.10 

Commi tment 

33.18 

Flexibility 

9.79 

Attention  to 

Detail  21 . 09 

Decisiveness  10.84 


Successful  Recr u i ter 


Demographic  Characteristics 


8.90 

Age  14.97 
Family  Support  24.67 
AFQT  54.59 
College  Experience  5 . 77 


Personality  Characteristics 


7.43 

Self-Image  8.45 
Integrity  35.98 
Extroverted  32 . 69 
Sense  of  Humor  16.72 
People-Oriented  6,17 


Spec  1 f ic  Experience 
15,88 

Sales  Experience  24.76 
Public  Speaking  Exp.  52.02 
Counselling  Exp.  23.22 
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Profile  of  the  Success  f ul 


Expert  #15. 
Communication  Skills 


24.16 

Public  Speaking 
Sklll3  13.80 

Vtiting  Skills  6.82 

Listening  Skills  20.68 

Informing  22.56 

Persuading  31.14 


Military  Background 
16.19 

Paygrade  24.97 

Years  of  Svc. 

(Act.)  61.83 

Years  of  Svc. 

(Res.)  13 . 20 


Behavior  Character ist ics 


13.81 

Self-Starter  26 . 66 

Commitment  9.05 

Flexibility  27.70 

Attention  to 

Detail  17 .88 

Decisiveness  18.70 


Recruiter 


Demographic  Character lstlcs 

16.70 

Age 

13.23 

Family  Support 

24.46 

AFQT 

43.19 

College  Experience 

19.12 

Personal lty  Character  1st ics 


22.08 

Self-Image 

13.62 

Integr i ty 

58 .32 

Extroverted 

7.85 

Sense  of  Humor 

4.39 

People-Or iented 

15.82 

Specific  Experience 

7.06 

Sales  Experience 

19 . 68 

Public  Speaking  Exp. 

31.57 

Counselling  Exp. 

48.75 
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Profile  of  the  Successful  Recruiter 

Expert  MRA 


Communication  Skills 

Demographic  Characteristics 

43.44 

4.88 

Public  Speaking 

Age 

10.71 

Skills 

14.72 

Family  Support 

26.95 

Writing  Skills 

LJ1 

AFQT 

50.75 

Listening  Skills 

27.14 

College  Experience 

U-60 

Informing 

17.42 

Persuading 

36.43 

Mi  1 i tar v  Backqr ound  Personality  Characteristics 


7.80 

19 . 37 

Paygrade 

46 .15 

Self-Image 

9.70 

Years  of  Svc. 

Integrity 

50.67 

(Act. ) 

49.76 

Extroverted 

25.04 

Years  of  Svc. 

( Res .  ) 

4.10 

Sense  of  Humor 

6.37 

People-Or iented 

8.21 

Behavior  Character  1st ics 

Specific  Experience 

18.86 

5.64 

Self-Starter 

31.24 

Sales  Experience 

34.58 

Commi tment 

26.84 

Public  Speaking  Exp. 

38.99 

Flexibility 

16.32 

Counselling  Exp. 

26.43 

Attention  to 

Detail 

16.92 

Decisiveness 

8.67 
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APPENDIX  D 

THE  EXPERT  SYSTEMS  EVALUATE  HYPOTHETICAL  RECRUITER  APPLICANTS 
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Candidate  Rankings  Based  cn  Expert  Systems 
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EVALUATION  OF  CONCEPTS 


TAGF  *  RECRUIT 


EXPERTS: -  #1 

CONCEPTS: -  COMMON  I  DEI10GRA  MI  L  IT  AR  PERSONA  BEHAVIO  SPEC  IF  I  OVERT.?  L 


-  -  - - 

IS.  9 

32.  7 

36.0 

27.  1 

28.  7 

44.5 

4  1  .  O 

Ei 

64.5 

51.2 

34.  2 

47.9 

47.5 
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L 
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Expert  I  2 
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*  *  f  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  -  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  SYS 1  EH 


BELOW  STANDARD  SUPERIOR 


EVALUATION  OF  CONCEPTS 


TALL  *  RECRUIT 


FXFERT3: -  #2 


CONCEPTS: - 

COMMUN I 

DEMOGRA 

MILITAR 

PERSONA 

BEHAVIO 

SPECIF  I 

OVERALL 

A 

t  1.0 

24.  1 

66.5 

1.0 

1.0 

19.9 

1.0 

R 

94.3 

75 . 6 

77.3 

67.5 

68.4 

99.0 

62  •  2 

n 

54.5 

53.6 

70.3 
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56 . 6 

53.  7 

54.7 

D 
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29.  4 
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47.8 

44.  2 

60 .  5 

F  . 

31.6 

50.9 
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43.6 

48.3 
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64.4 

54  •  7 

48.2 

52.  1 

58.4 
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42.7 

52.4 
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K 

6.5 

34.6 

62.3 

8.  7 

12.  1 

26.  7 
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L 
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62.6 

69 . 3 

77.  3 

59 .  3 

n 

80.3 
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74.6 

46.  1 

68.7 

5  j  m  M 

60.7 

ri 

1.0 

1 . 0 

67.7 

11.0 

1 .  o 

57.  / 

1 . 0 

0 

60  •  3 

35.7 

30.0 

53.7 

54.  3 

44.2 

53.  3 

P 

54.0 

69.  4 

82.6 

38.3 

37.3 

59.7 

56.6 

s 

63.9 

73.8 

71.6 

58.  9 

82.  4 

63.9 

52.6 

T 

56.5 

34.5 

79.  7 

17.9 

21. 7 

-• 

u  /  .  :• 

'•***  n 

U 

21.2 

43.  9 

73.8 

10.7 

49.  1 

24.8 

31.5 

V 

25.  4 

37.  1 

78.8 

57.5 

41.0 

99.  0 

,ir  t 

J.  ■-> 
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*  *  ♦  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  *  PROFILE  OF  THE  SUCCESSFUL  RECRUITER 


EXPERT  SYSTEM 


liligisi  h 


..^.a . . m**- 

lilliSIhllili42- 1 

ll8§iSi4i8^1l34o.a 


mkl  7-5 

g,1-^ 


BELOW  STANDARD 


60—i—  70— *- 
SUPERIOR 


go—1—  90 


EVALUATION  OF  CONCEPTS 


TASK  *  RECRUIT 


EXPERTS: - 

CONCEPTS: - 


A 

If 

C 

D 


I 
J 

f;. 

L 

II 
f! 
0 
P 
S 
T 
U 


#J 


CDMMUNI 

DEMOGRA 

MIL  I  TAR 

PERSONA 

BEHAVID 

SPECIF  I 

OVERALL 

*">  '“v 

21.5 

43.3 

1.5 

1 . 0 

0.3 

1 . 0 

79.  4 

60.5 

83.8 

67.  1 

7  4 .  6 

03.8 

74.3 

4S.0 

50.5 

■  39.  Q 

46.9 

48.2 

52.  Z 

54.  3 

34.7 

31.4 

46.7 

48.  4 

23 . 5 

51.6 

42.  1. 

08.4 

50.  8 

42.2 

53.6 

33.8 

28.8 

\j  ! .  6 

30.  1 

4  7.7 

38.4 

•j '  -7  .  ^3 

37.0 

30.9 

.  2 

24.2 

60.7 

67.  4 

18.5 

31.1 

46.7 

28.  3 

37.9 

45.  1 

38.5 

37.9 

38.2 

42.4 

44.0 

62.  8 

55.  4 

58.  7 

41. 1 

57.6 

51.0 

60. 0 

56  •  6 

29.  7 

40.5 

53.9 

54.7 

50.  3 

60. 5 

15.  O 

30.6 

41.5 

15.  2 

14.2 

20.4 

13.o 

65.5 

59.8 

56.2 

60.  1 

64.  1 

69 . 6 

68.  6 

63.  4 

60.5 

77.2 

67.0 

45.3 

51.6 

A3.o 

11 .  a 

13.  4 

37.3 

1.0 

24.9 

I.)-  >  •  3 

7  •  .“j 

52.  a 

46.  8 

40.  6 

47.8 

46 .  a 

47.  1 

rr 
»..*  J  • 

53.8 

51.5 

41.3 

48.3 

.39.  1 

50.  3 

53.  9 

47.5 

54.7 

93.  4 

55.7 

57.  4 

47.  3 

53 .  *f 

51.1 

52 .  8 

54.4 

—? 

✓ 

34 . 6 

50 . 0 

43 .  2 

C" 

/  ■  <J 

33.0 

64.5 

33.  9 

40.  1 

17.6 

40.8 

30.7 

31.6 

59.6 

46.  5 

42.  3 

68.  1 

48.8 
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*  *  *  CONCEPTUAL  LEVEL  *  *  » 

CONCEPT  «  PROFILE  OF  I HE  SUCCESSFUL  RECRUITER  EXPERT  S/SIEI1 


EVALUATION  OF  CONCEPTS 


1ASL  =  RECRUIT 


EXPERTS: -  M 

CONCEPT  S: -  Cut  U1UN I  DEMOGRA  NIL  I  TAR  PERSONA  BEHAVIO  SFECIFI  OVERALL 


A 

1.0 

33.7 

47.2 

4.  1 

CT  -T 

U  m  ^ 

'"'rj 

.  4t- 

9.  1 

B 

as.  7 

40.  2 

67.7 

59.7 

66.  1 

33.0 

53.  t 

C 

55 .  5 

57.  0 

56.4 

49.  V 

53 . 3 

49.6 

52.  4 

D 

59.7 

35.  O 

60. 0 

50 .  u 

44.7 

49.  1 

41..  2 

E 

60  •  O 

64 . 2 

46.0 

511.  7 

39.5 

40.  o 

r  •  - 

•  J  ■  .  -• 

F 

54.6 

41.2 

53.6 

26.9 

40.  7 

37.0 

4  1.1 

& 

31.0 

57.7 

52.7 

o  '•*»  *• 

49.  7 

45.'.' 

40.  3 

H 

44.  1 

54.9 

53.  3' 

40.  9 

44.7 

43.2 

40 . 0 

I 

!  70.2 

56.2 

54.6 

Cr ■?  *T 
bJ  . 1  ■  <J 

50,  1 

36.4 

53. 3 

J 

55.9 

43.  0 

55.9 

44. 5 

52.  5 

48.  9 

49.0 

l 

16.2 

43.9 

44.  1 

18.5 

20.  7 

J*.  •  J 

26.9 

L 

7  j  .  0 

54.5 

65.  9 

53.5 

64.2 

64.8 

U  <j  • 

;  i 

71.  1 

58. 9 

5o .  5 

51.2 

42.9 

40.  2 

«.  •  bj 

!) 

7.9 

14.6 

55.  1 

1  5.  1 

70.  7 

53 . 6 

15.  3 

c 

59.4 

53.  7 

61.7 

43.  7 

54 . 2 

46.  4 

U  •  2. 

r* 

51.4 

55.5 

6  .<  •  -J> 

55.  1 

43.8 

66 . 0 

51.7 

5 

57.7 

59 .  O 

40.5 

59 . 0 

64.7 

47.6 

! 

57 . 2 

46.7 

54.3 

'ncr 

_  wi  •  4. 

46.2 

59.6 

‘13.5 

U 

30.  7 

47.  6 

6  3.o 

23 . 7 

:a.  6 

30 . 2 

11.6 

V 

34.7 

S3  •  2 

63 » 3 

41.5 

49.6 

77.  Z 

■1 7  •  6 
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*  *  *  CONCEPTUAL  LEVEL  *  *  * 

- CONCEPT  =*  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT 


Expert  #  5 

L 

I 

Et 

C 

U 

s 

E 

P 

D 

H 

M 

F 

J 

T 

G 

V 

U 

K 

N 


BELOW  STANDARD  SUPERIOR 


EVALUATION  OF  CONCEPTS 


T  ASP:  =  RECRUIT 


EXPERTS: - 

CONCEPTS: - 

#5 

conriUNi 

DEMOGRA 

NIL I TAR 

PERSONA 

BEHAVIQ 

SPEC  I F I 

OVERAL 

M 

1 . 0 

14.  1 

46.7 

1 . 0 

1 . 0 

1 7 . 3 

0.5 

B 

70.5 

65.  1 

51.4 

65.0 

72.  7 

92.8 

50 . 5 

C 

54.7 

64.  1 

59.4 

49.8 

55.  1 

46 .  S 

47.0 

D 

33 « 9 

50.9 

64.9 

cr  4  nr 

1  •  wJ 

43.  7 

52.  1 

4  1 . 0 

E 

65.  a 

53.3 

48.  8 

60 .  5 

33.  1 

24 . 0 

42.  7 

F 

-  j*  ■  >1> 

57.0 

72.  5 

26.  1 

42.  9 

30.3 

36.9 

G 

30.3 

62.3 

40.  4 

13.5 

45.  5 

45.0 

7*'  3 

H 

43. a 

55  •  j 

53.2 

38.9 

44.0 

37.4 

39.  ? 

T 

63.0 

73.5 

63.4 

52.  1 

53 .  2 

51.0 

51.3 

j 

42.0 

29.3 

64.6 

49.2 

59.3 

60 . 0 

35.  9 

K  - . . 

13.7 

30.  3 

50.2 

8.5 

13.3 

22.  V 

13.  7 

L 

68.2 

75.8 

62.9 

63.  1 

69.3 

67.  4 

ETC;  J 

wl  fcj  a  C3 

M 

56.  7 

50.  1 

31.9 

55.7 

37.2 

27.7 

38.0 

N 

8.5 

3.8 

70.5 

1 . 0 

29.  3 

69.2 

to.  3 

0 

57.  0 

57.3 

69.0 

49.  1 

54.8 

44.3 

46.3 

D 

29.4 

50.5 

72.6 

55.4 

46.9 

57.3 

41.9 

s 

59.8 

58.8 

26.7 

59.  1 

68.  7 

64.4 

44.9 

T 

49.  7 

46.3 

50.8 

21.1 

51.4 

51.5 

34.3 

U 

“T  '-t 

23.  1 

58.2 

27.  1 

34 . 0 

16.  8 

24.  1 

*  1 

21.3 

74.6 

39.  2 

45.6 

—j  1  ET 

/  1  .  u 

29  •  5 
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t  *  *  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  *  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  SYSTEM 


k:r3  >•!»"!  !:i  'a  !i!  "i  1.1 1"  (if  - 

FELON  STANDARD 


SUPERIOR 


t 


EVAI.UAT 1 01 1  OF  CONCEPTS 


task  =  RECRUIT 


EXPERTS: -  #6 


CONCEPTS: - 

CUMMUNI 

DEMOGRA 

MI  LI  TAR 

PERSONA 

BEHAVIO 

SPECIF I 

OVERALL 

A 

1 .0 

34.9 

47. 8 

2.  1 

1.0 

30.  1 

1.0 

p 

90.  1 

04 . 4 

07.3 

71.7 

75  •  3 

99.0 

73.  7 

c 

60.6 

50.7 

55.3 

57.5 

53.7 

51. 1 

62 . 6 

p 

4  4 . 0 

47.  1 

62.3 

58.  3 

31.9 

53.6 

5?.  1 

£ 

75.2 

57.8 

44.8 

63.0 

34 . 3 

39.6 

61.4 

F 

4 1 . 0 

57.2 

53  •  3 

38.  1 

43.9 

37.  1 

40.5 

G 

32.0  • 

71.8 

66.5 

23.9 

52.4 

46.5 

42.  3 

H 

48.5 

46. 0 

51.3 

47.5 

43.3 

41.6 

53 . 5 

I 

73.3 

62.2 

60.  S 

58.7 

57.8 

68.0 

66  •  5 

J 

56 . 3 

56 . 3 

53.8 

60 . 3 

48.0 

56.9 

59 . 3 

J  ' 

18.6 

38.0 

41.3 

19.  8 

16.  8 

31.  1 

20.6 

L 

79.2 

68.9 

72.6 

69.  7 

68.  4 

78.5 

72.  1 

M 

78.5 

76.  5 

63.  0 

56.3 

33  n 

54.4 

58.2 

M 

7.8 

14.8 

47.6 

17.3 

40.  2 

72.  1 

16.  1 

n 

66.  1 

49.3 

60 . 3 

55.  7 

58.  3 

43.  1 

64 . 9 

p 

55 . 2 

59 .  3 

60.8 

54.2 

48.0 

90.  1 

59.  7 

5 

71.9 

63.2 

67 . 8 

63.  1 

58.8 

59.8 

67.6 

T 

59.7 

34.  6 

61.1 

39.5 

47.  4 

71.3 

5  3.8 

U 

32.  1 

47.  7 

77.3 

42.4 

33.8 

30.  1 

4  4 . 6 

V 

31.6 

45.5 

75.  5 

46.3 

65  •  3 

nn 

7  -1  • 

c-  er 

« J 
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CO  H 


EVALUATION  OF  CONCEPTS 

TASK  =  RECRUIT 

EXPERTSj -  #MAGR 

CONCEPTS: -  COMMUMI  DEHOGRA  MILITAR  PERSONA  BEHAVIO  SPECIF!  OVERALL 


A 

I  1.0 

33.7 

50.5 

1.2 

1.0 

9.0 

1.0 

B 

I  636 . 5 

7  3  •  O 

74.6 

65.0 

63.  5 

86 . 3 

73.4 

U 

54.4 

57.6 

53.  7 

52.7 

54.0 

55.4 

61.7 

D 

!  36. 4 

51.6 

55.9 

59.4 

40.5 

4  4.9 

51.4 

E 

66.2 

56.4 

45.  4 

62.9 

38.2 

2 .  .  •!> 

64.5 

F 

31.6 

61.7 

41.6 

30.7 

40.7 

37: „  7, 

42.8 

e 

27.5 

66.  2 

67.8 

20.5 

45.7 

46.  7 

37.  4 

H 

42.  4 

53.  1 

49.6 

43.  7 

44.  1 

45.2 

57.0 

I 

70.8 

67.2 

42.9 

51.0 

51.3 

60 . 9 

6  6.9 

J 

57 .  a 

41.5 

48.  0 

52 . 5 

57.8 

39.4 

62 . 5 

K 

13.7 

41 . 1 

39.  7 

17. 3 

16.6 

22.0 

1 S.  5 

L 

i  73. 6 

71.8 

66. 6 

63.  9 

66.3 

72.7 

69.  ? 

M 

65.  7 

66 . 3 

73.  9 

63.  8 

40 .  6 

34.4 

6  3 . 2 

M 

12.  1 

8.  4 

31.2 

3.  1 

'■n  -y 

54.6 

1 0 .  4 

0 

59.  6 

53  ■  3 

52.  4 

51.0 

53.4 

52.  1 

63.  4 

P  j  55.2  64.6  46.7  63.4  44.1  65.6  62.3 

54.9  61.2  76.0  37.4  63.3  40.9  60.6 

57.1  46.8  61.3  24.9  30.5  62.8  55.9 

U  31.7  48.3  69.0  34.6  35.0  16.0  43.2 

V  31.4  43.6  47.2  37.6  43. 7  64.5  46 . 2 
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*  *  *  CONCEPTUAL  level  ♦  *  ♦ 

CONCEPT  *  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  SYSTEM 


Expert  #  PA3R 

B 

S 

I 

0 

C 

E 

J 

(1 

P 

D 

r 

H 

F 

V 

U 

G 

K 

N 

A 


BELOW  STANDARD  SUPERIOR 


EVALUATION  OF  CONCEPTS 


TASK.  =  RECRUIT 


EXPERTS: -  *PAGR 


CONCEPTS: - 

COITNUN I 

DLI 1U0RA 

MIL  I  TAR 

PERSONA 

REHAV I 0 

SPEC IF I 

OVERALL 

A 

1 . 0 

34 . 9 

46.  7 

2.  1 

1.0 

17.3 

1.0 

8 

90.  1 

34.  4 

51.4 

71 , 7 

7  2 . 7 

72.8 

71.7 

c 

6*  * .  6 

50 . 9 

59.4 

cr  ~y  er 
-J  •  -J 

55.  1 

46  •  3 

63.  4 

D 

44.0 

43.  1 

64  m  *7 

53.  3 

43.  7 

52.  \ 

55.  0 

E 

75.2 

53.3 

49.3 

63.0 

33.  1 

2  4.0 

62.3 

F 

4  1 . 0 

57.2 

72.5 

33.  1 

42.9 

30 .  .3 

47.  4 

G 

32.0 

71.9 

40.4 

23.9 

45.5 

45.0 

33.7- 

H 

•18. 5 

46.0 

53.2 

47.5 

44.0 

37.4 

54.4 

I 

7  ^ 

62.2 

63  *  4 

59.7 

53  .  2 

51.0 

62  •  7 

J 

56.  3 

36.3 

64.  6 

60 .  3 

59.3 

60 . 0 

62.6 

f 

13.6 

38.0 

50.  2 

19.3 

13.3 

22.9 

2 1 . 0 

L 

77.2 

68.9 

62.9 

69.7 

69.3 

6  7-4 

7  i  .  G 

1 1 

73.5 

76.5 

31.9 

56.3 

37.2 

27.7 

53.7 

M 

7.3 

14.8 

70.  5 

17.3 

2°.  a 

69.2 

13.  7 

0 

66.  1 

49.3 

69.0 

55.  7 

54.  3 

44.3 

64.5 

P 

n  c  'i 
~J— *  •  -L 

59.3 

72-6 

54.2 

46.9 

57.3 

53.  7 

s 

71.8 

63.  2 

26.7 

63.  1 

63.9 

64.4 

66-5 

T 

3  •  7 

34. 6 

50 . 3 

39.5 

51 . 4 

51.5 

54.5 

U 

32.  1 

4  7.3 

53.  2 

42.  4 

54.0 

16.3 

45.  3 

V 

31.6 

45.5 

74.6 

46.3 

45.  6 

71.5 

4  7.6 
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*  *  *  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  =  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  SYSTEM 


BELOW  STANDARD  SUPER I MR 


EVALUATION  UF  CONCEPTS 


TASK  =  RECRUIT 

EXPERTS: -  #8 

CONCEPTS:- - .COMMUNI  DEMOGRA  MIL  I TAR  PERSONA  BEMAVIQ  SPECIFl  OVERALL" . 


A 

13.7 

32.7 

36.0 

27.  1 

23.  7 

44.5 

41.0 

B 

64.5 

51.2 

34 . 2 

47.9 

4  7.5 

47.5 

49.5 

C 

45.  1 

41.7 

43.7 

43.  3 

43.5 

'’n  er 

•v  y  • 

17.9 

D 

•  4 

*a.  i 

40.  3 

40.  6 

41.3 

4  3.4 

46.  : 

E 

54.0 

42.3 

36.7 

49.6 

36 . 0 

49.5 

4  0 . 6 

F 

30. 6 

39.  1 

46.  1 

.•  ■-*«  6 

42.6 

30.4 

45.0 

G 

33.0 

46.7 

26.  1 

34.0 

39.  1 

37.7 

46.  3 

H 

39.2 

39.5 

42.5 

40 . 3 

40.  B 

39.  4 

46.  7 

I 

55 .  5 

47.3 

36.  2 

44.4 

44.3 

46.  1 

49.  O 

J 

46 . 6 

34.5 

42.  1 

4o.  a 

46.  5 

41.2 

4  6.9 

K 

26.  1 

35 . 0 

37.2 

-'a-  *  _ 

33.  4 

41.9 

43.3 

L 

55.  7 

47.  1 

41.1 

47.0 

46.9 

42.7 

49.2 

M 

49.0 

46.  7 

23.6 

43.  1 

34 . 3 

51.5 

4  7.9 

N 

30 .  2 

47.7 

31.  S 

41.9 

33.6 

42.6 

0 

46.8 

40.  1 

44.3 

41.3 

42.3 

39.  1 

4  / .  6 

P 

42.5 

41.5 

45.  B 

47.5 

42.  7 

r  i  «r- 

jl  •  j 

47.9 

S 

50.0 

4  7 . 0 

45.9 

46.5 

38 . 5 

43.  1 

T 

46.7 

33.3 

32.9 

36.  7 

43.  0 

45.  7 

4  7.0 

U 

35.  1 

36 .  4 

35.6 

37.9 

~*er  *-\ 

44.2 

15.  1 

V 

37.  1 

_-5  .  6 

41.3 

-•5 .  2 

36 .  S 

43.2 

45.0 
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*  *  *  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  *  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  3  r'Sl  Ert 


BELOW  STANDARD  SUPERIOR 


EVALUATION  OF  CONCEPTS 


TASK  ~  RECRUIT 


EXPERTS: -  #9 

CONCEPTS: -  CON, NUN I  DEMOGRA  MILITAR  PERSONA  BEHAVIO  SPEC IF  I  OVERALL 


A 

B 

C 

D 

E 

F 

G 

H - 

I 

J 

K 

L 

M 

N 

0 

P 


1 .  o — 

-  14.1 

46.  7 

1.0 

1 . 0 

17.3 

3.5 

70.  5 

65.  1 

51.4 

65.0 

72.7 

92.3 

50 . 5 

54.7 

64.  1 

59.4 

49 . 3 

55.  1 

46.3 

47.  0 

33.9 

50.  9 

64. 9 

51.5 

43.7 

52.  1 

4  1 . 0 

65.9 

53.3 

43.3 

60.5 

33.  1 

24.0 

42.  7 

■  J 

57.  0 

T"N  rr 

/  m 

26.  1 

42.  9 

30.  3 

36 . 9 

3  0  •  3 

62.3 

40.4 

13.5 

45.5 

45. 0 

T  T  T 

43.3 

tr  tr  *■*--  -  - 

«  ■-» 

—  -  erpj"  ‘n 

38.9 

44.0 

37 . 4 

39.9 

63.0 

73.5 

63.4 

52.  1 

e** 

sJ 

5 1 . 0 

51.3 

42.  0 

29.3 

64.6 

49.  2 

59.9 

60 .  < » 

.  9 

13.  7 

30 . 3 

50.2 

3.5 

13.8 

"22.  9 

X  •  j  •  / 

68.2 

75.3 

62.9 

63.  1 

69.3 

67.4 

«j  o  •  6 

56.7 

50.  1 

31.9 

55.  7 

37.2 

-v-v  -r 

42  /  -  / 

30 . 0 

9.5 

3.9 

70.5 

1 . 0 

29.3 

6  9  .  *1 

10.  3 

57.0 

57.9 

6  9  •  0 

49.  1 

54.  S 

44.3 

•To  •  •.> 

29.4 

50.  5 

72.  6 

55.4 

46.9 

wJ  I  •  •  _> 

4  1 . 9 

59.  3 

58.  S 

26.  7 

59.  1 

63.9 

64. 4 

44.  9 

49.  7 

4  6  •  3 

50. 3 

21 . 1 

51.4 

51.5 

34.3 

3 

23.  1 

58.2 

27.  1 

34 . 0 

16.9 

24.  1 

33.3 

2 1 . 3 

74.6 

39.2 

45.6 

71 .5 

•.  r~ 

7  a  vJ 
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*  f  *  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  «  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  STS I EM  *• 


EVVlLUr.l  IIJIJ  UP  LOML'EPIS 


TASK  ~  RECRUIT 

HXPEFvTS: -  <10 


CONCEPTS: - 

contiUN  i 

OEi-IUGF.A 

MI  LI  TAR 

PERSONA 

BEHAVIO 

SPECIFI 

OVERALL 

A 

1.0 

“•  rr  ~y 

•  / 

47.  2 

4.  1 

cr  -* 

•J  «  ■  j 

28.2 

9.  1 

B 

83.  7 

40.  2 

67.7 

59.  7 

66.  1 

33.0 

33.  1 

c: 

55  •  5 

57.  3 

56  •  4 

48,  9 

53.5 

49.6 

52.  4 

D 

39.  9 

75.0 

60.0 

51 1 .  o 

44.3 

49.  4 

43.2 

E 

68.0 

64  •  2 

46.0 

53.  7 

39.3 

40.0 

wj  ..."  •  -J 

F 

34.6 

41 . 2 

58 .  6 

26.9 

40.  7 

37.0 

41. 1 

G 

31.0 

57.7 

rr  --N  -J 

m  t 

47.7 

45.0 

TO.  6 

H 

44.  1 

54.9 

55.3 

40.  9 

44.7 

43.2 

43.0 

I 

70.  2 

56.  2 

54.6 

53  •  5 

30.  1 

56.  4 

•I 

55.  9 

43.0 

55.9 

44.5 

52  •  5 

48.7 

49.0 

K 

16.2 

47.  9 

44.  1 

18.5 

20.  7 

Tn  cr 

26.9 

73.0 

54.3 

65.  9 

rr  n  rr 
JO.  J 

64.2 

64.8 

33.  0 

n  i 

71. 1 

53.  9 

cr er 

vj'J  •  wJ 

51.2 

42.7 

48.  2 

c~  —>  c— 

-J  ...  • 

N 

7.9 

14.6 

55  •  1 

13.  1 

28.  7 

57.6 

15.5 

D 

59.  4 

53.  7 

61.7 

40.7 

54.2 

46.4 

e  »  -* 

•J  « 

P 

5 1 . 4 

55 . 5 

63. 3 

55.  1 

47.3 

66 . 0 

51.3 

3 

5? .  7 

59  .  o 

43. 5 

57.0 

64.7 

49.6 

53.6 

7 

er  -r 

U  .  . 

46.7 

54.5 

C~  '  -1 

46.  2 

59.6 

45.5 

U 

30.  7 

4  7.6 

63 . 6 

23.7 

38.6 

30.  2 

41.6 

\* 

34.  7 

6S.3 

4  t .  5 

49.6 

72.  3 

42.  6 
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*  *  t  CONCEPTUAL  LEVEL  *  *  « 

CONCEPT  =  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  SYSTEM 


EVAI.IIA1  Mill  OF  C8NCFFTS 

7  ASF  =  RECRUIT 


EXPERTS: -  #11 


CONCEPTS: - 

CQMMUN I 

DEMOGRA 

MILITAR 

PERSONA 

BEHAVIO 

SPEC IF I 

OVCRAL 

#1 

^  1 . 5 

43.  3 

1  •  5 

1 . 0 

9" . 

1 . 0 

B 

79.4 

60.5 

83.8 

67.  1 

74.6 

03.8 

74.3 

C 

43.0 

50 . 3 

39.8 

46.9 

43.2 

cr  r-\ 

*J  —  •  -• 

54.  3 

D 

34.7 

31.4 

46.  7 

48.  4 

-■>  -»  c~ 

Jf  >  .1 

54.6 

4  2.1 

E 

58. 4 

50.3 

42.2 

5  •  6 

33.0 

28.8 

54 . 6 

F 

30.  1 

47.7 

38.  4 

33  •  2 

37.0 

30 . 9 

38.2 

G 

34.2 

60.  7 

67.4 

13.5 

31.1 

4  6.7 

— v  t~\  -r 

O  . 

H 

37.9 

45.  1 

38.5 

37.9 

33.2 

42.  4 

44.0 

I 

62.8 

55.4 

58.  7 

41 . 1 

57.6 

51.0 

60 . 0 

■  1 

56.  6 

29.  7 

40.5 

to  3 . 9 

54.7 

50.3 

60 . 5 

K 

15.  0 

30.  6 

41.5 

1  u  •  3 

14.2 

20.  4 

1  3 .  0 

L 

65.5 

59.8 

cr  t  -~i 

JO  .  — 

6U  •  1 

64.  1 

69.  6 

69.6 

n 

63. 4 

6 U.  5 

77.2 

67.  0 

45.3 

31.6 

67.  ( ) 

M 

11.8 

13.  4 

37.  3 

1 . 0 

24.9 

53.0 

/ , '  j 

0 

52.8 

46.3 

40 .  6 

47.3 

46.  0 

47.  1 

p 

53 .  9 

51.3 

41.3 

48.3 

39.  1 

50.  3 

CT'»  O 
•J  -•  .  7 

s 

47.5 

54 .  7 

83.  4 

cr  e-  -7 

57.4 

47.3 

r-r->  o 
,.J  v. )  •  7 

T 

51.1 

-  O 

54.4 

34.o 

50 .  '7 

43.2 

u 

27.5 

33.  8 

64.5 

33.9 

40.  1 

1 7 . 6 

40.3 

V 

30.  7 

5l.o 

59.  6 

46.5 

42.3 

68.  1 

48.3 
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*  *  *  CONCEPTUAL  LEVEL  *  *  * 

CONCEPT  =  PROFILE  OF  THE  SUCCESSFUL  RECRIJI1ER  EXPERT  5. 3TEI1 


,\o .  o 


A (_) - ■  ^  / (J— Lw 

SUPER I UR 


30  _i_  90 


F VALUATION  UE  CONCEPTS 


TASt  =  RECRUIT 


EXPERTS: -  #12 


CONCEPTS: - 

COMNUMI 

DEMOGRA 

NILITAR 

PERSONA 

BEHAVIO 

SPECIF  I 

OVERAl 

A 

1.0 

24.  1 

66.5 

1 . 0 

1.0 

7.0 

1.0 

D 

94.3 

75.6 

77.3 

67.5 

27.5 

79 . 3 

4  1 . 0 

C. 

54.5 

58.6 

70.3 

49.2 

44.  1 

63.3 

•10.  : 

D 

36.0 

29.  4 

oo.  a 

38.5 

13.7 

38.3 

-•>  »  rr 
.  .  •  .  .J 

E 

67  •  Q 

64.3 

rr  ~r 

sj  -•  .  •_> 

—» •*  »-* 

1  5 * 6 

■V'.'l 

F 

31.6 

50.  9 

75.8 

25 • 5 

27 . 5 

43.4 

76.0 

G 

24.2 

80.7 

77.3 

17.4 

55 .  4 

11.! 

H 

40.2 

51.8 

60.  3 

37 . 4 

57.0 

55 . 3 

33.  1 

I 

74.6 

60.5 

64.4 

54.7 

Z  3  -  6 

64.7 

17.9 

J 

51.3 

54.  1 

/  O  •  _» 

42. 

41.1 

33.  4 

43.  5 

Y. 

6.  5 

34.6 

62.3 

3.7 

9.3 

oer  rr 

.  U 

1.0 

L 

77.8 

74.0 

05.  1 

62.6 

44.8 

76  -  B 

49.  2 

M 

80.5 

86.2 

74.6 

46.  1 

26.  4 

23.  1 

42.  3 

N 

1 . 0 

1 . 0 

67.7 

11.0 

1.0 

64.2 

1 . 0 

0 

60.5 

ercr  -* 
wJ-J  •  / 

80.0 

53.7 

40.  9 

63.0 

47.3 

P 

54.0 

69.  4 

82.6 

53.3 

30.  9 

65.  3 

T 

*7  » 

3 

63.9 

73.  a 

71.6 

53.  9 

41.2 

42.5 

48.  5 

T 

56.5 

34.3 

79.7 

17.9 

29  -  6 

63.9 

40.  5 

U 

31 .2 

43.8 

7  7.  H 

10.  7 

27.3 

14.2 

r-)  rr 
*  7  m  .J 

V 

25.  4 

37.  1 

70.3 

57.5 

17.9 

33  •  1 

15.0 
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*  *  *  CONCEPTUAL  LEVEL  *  $  *  '  - 

CONCEPT  =  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXF'FRT  SYSTEM 


S  i.U 

BELOW  STANDARD 


50- 


60—1 — 

SUPER: !  OR1 


TASK  =  RECRUIT 


EVALUATION  of  concepts 


EXPERTS s - 

CONCEPTS: - 

A 

B 

C 

D 

E 

F  l 

6 

H 

I 

J 

K 

L 

M 

N 

0 

P 

3 

T 

U 

u 


#13 

CQMtTUN  I 


77.4 
48. 0 

34 . 7 

53.4 
30.  1 
24.  2 
37.  9 

62.  a 

56 .  6 
15.0 

65.5 

63.  4 
11.3 

52.8 

53.8 

47.5 
51 . 1 

'“tT  r- 
-  /  .  ^  J 

30 .  7 


DEMOGRA 

MI LI TAR 

PERSONA 

BCHAV10 

SPEC  IF 2 

OVERALL 

21.5 

43.  a 

1.5 

1.0 

0.3 

1 . 0 

60 . 5 

83.8 

67.  1 

74.6 

83.  3 

74 . 3 

50.  5 
31.4 

39.8 

46.7 

46.9 

43.  4 

48.  2 

O  T  C7 
yJt 

c-  r 

54  •  6 

r-  . 

J  1. 

42.  1 

50.8 

42.  2 

53.6 

33.  8 

23.  8 

5.1  _  6 

47.7 

38.4 

T7  «*n 

37.0 

30.  9 

33.  2 

60.  7 

67.4 

18.5 

31 . 1 

46.7 

28. 3 

45.  1 

33.5 

37.  9 

38.2 

42.  4 

4  4 . 0 

55 . 4 

58.7 

41.1 

57.6 

51 . 0 

60 .  O 

29 .  ’ 

40.5 

53.7 

54 . 7 

50.3 

6* ) .  5 

' ' .  6 

41.5 

15.2 

14.2 

20.  4 

1 3 . 0 

-.9.8 

56.  2 

60.  1 

64.  1 

69.6 

63.  6 

60.  5 

77.2 

67.0 

45.3 

51.6 

63.  0 

13.4 

37.3 

1.0 

24 . 7 

53.  0 

7.5 

46.8 

40.  6 

47.  8 

46.  0 

47.  1 

crfr  -1* 

.  j  VJ  .  j, 

51.5 

4  1 . 3 

48.3 

39.  1 

50.  3 

53.  ? 

54 . 7 

83.  4 

55.  7 

57.  4 

47.3 

53.  7 

32.8 

33.0 

54.  4 
64.5 

23.7 

33.7 

34.6 

40.  1 

50 . 0 

17.6 

43.2 

4o.  n 

■1.6 

57.6 

46.5 

42.  3 

68.  1 

•10.0 
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1  t  t  CONCEPT  UAL  LfVEL  *  *  » 

CONCEPT  *  PROFILE  UF  THE  SUCCESSFUL  RECRUITER  EXFFRT  SV'SIOI  ~ 


60— 1 —  70-4— 
SUPERIOR 


80—1 —  VO 


EVALUATION  UF  CUNCERTS 


TASK  =  RECRUIT 


EXPERTS: - 

CONCEPTS: - 

#14 

LOMMUNI 

DEMOGRA 

MIL  I TAR 

REPSONA 

BEHAVIO 

SPEC IF I 

OVERALL 

A 

1 . 0 

24.  1 

66.5 

1.0 

1.0 

7.3 

t.O 

B 

V4.3 

75 . 6 

77.3 

67.5 

27.5 

79.8 

41.0 

C 

54.5 

58.  6 

70 . 3 

48.2 

44.  1 

63.  S 

■13.  3 

D 

36.0 

29.4 

90.8 

33.5 

18.  7 

33.5 

■>  -»  •  • 

_•  .  ■  • 

E 

72.2 

67  •  B 

64.  8 

l~  *  •* 

•J  i  .* 

15.6 

6' 0 

F 

31.6 

50.9 

75.  8 

25  •  5 

27.5 

43.4 

26.  r< 

G 

n  i  n 
*T  •  A. 

SO.  7 

77.3 

17.4 

—} 

rc  4 

JJ  •  * 

14.  1 

H 

40.  2 

51.3 

68.  3 

37.4 

37.0 

53.5 

33.4 

I 

74.6 

6R.  5 

64.4 

54.7 

33.6 

64.9 

47.9 

J 

•J  1  •  O 

.34 .  1 

75.3 

42.  7 

41.1 

33.4 

43.5 

K 

6.5 

34.6 

62.3 

3.  7 

9.3 

— *cr  r~ 
«.J*  J 

1 . 0 

L 

77. a 

74.0 

95.  1 

62.  6 

44.8 

76.9 

47.  2 

tl 

80 . 5 

86.  3 

74.6 

46.  1 

26.  4 

23.  1 

4  2.3 

M 

1 . 0 

1 .  0 

67.7 

11.0 

1  . ') 

64 . 2 

t.'» 

0 

60.5 

55.  7 

30.  0 

53.  7 

40.9 

63.0 

4  7.5 

P 

54.0 

69.  4 

82.6 

53.3 

30 . 9 

65.8 

39.  3 

3 

63.9 

73.9 

71.6 

59.9 

41 . 2 

42.5  * 

43.  5 

T 

56.5 

34.5 

79.7 

17.9 

29.6 

63.9 

40.3 

U 

21.2 

43.9 

73.0 

10.7 

27.3 

14.2 

79.5 

V 

25.  4 

37 .  1 

79.0 

57.5 

17.9 

53.  1 

13.0 
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*  *  *  CONCEPTUAL.  LEVEL  *  *  * 

CONCEPT  «  PROFILE  OF  THE  SUCCESSFUL  RECRUITER 


EXPERT  SYS I EM 
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#15 


D  <H  IM  .«•  J  - 


I  U  b  h  I?  S  w  «•  is  if  1 1*,  w  4  is  b  si « I  13  lit  &  is  is  is  i.i  is  $  is « I  Is  P(  y  i;i  !il ;  1.1 ll i«  '  • 

i s 3 i M » ■!•  u  it !! 1 * s is* s' *■  -  ••  •’ '  •■- -  ■  ■  -  ■ 


“  J  a  I;  1  |  ll  S  B  iJ  I  f!  a  I  r!  if  1 1 S  ill !!  0 1!  ’ : 

Silks!  *  8118  LliiiliiJiiylJll! 


20—1—  30 — 1 

.C1W  STANDARD 


—  60—1 —  70-1- 
SUPERJOR 


EVALUATION  OF  CONCEPTS 


TASK  =  RECRUIT 


EXPERTS: - 

#15 

CONCEPTS: - 

COMMON I 

DEMOGRA 

MI  LI TAR 

PERSONA 

BEHAVIO 

SPEC IF I 

OVERALL 

A 

1.0 

34.9 

47.8 

2.  1 

1.0 

30.  1 

1 . 0 

B 

90.  1 

84.4 

37.3 

71.7 

75  •  3 

99.0 

“r  ~r  '•> 

/ 

C 

60.  6 

50.9 

55  •  3 

57  •  5 

53.7 

51. 1 

62  •  6 

D 

44.0 

43.  1 

62.  3 

58.3 

51.9 

5  -• .  0 

57. 1 

E 

75.2 

53.8 

44.0 

63.0 

*■  ‘1  •  •-> 

-*r  r-y  j 
,•  T  •  « 

61.4 

F 

41.0 

57.2 

53.8 

38.  1 

43.9 

37.  1 

in  cr 

Hb.  -J 

G 

32.0 

71.8 

66 . 5 

23.9 

52.4 

46.  5 

4  2.  3 

H 

48.5 

46.0 

J  1  •  O 

47.5 

43.3 

41.6 

53 . 5 

I 

73.3 

62.2 

60.  Q 

58.7 

57.3 

60.0 

no .  -J 

J 

56.  3 

36.8 

53. 0 

6' s .  3 

48.0 

56.  V 

*jV  .  ...» 

K 

18.6 

38.  0 

41.3 

19.  8 

16.3 

31.1 

20 . 6 

L. 

79.2 

68.9 

72.  6 

69.7 

68. 4 

7o .  5 

72.  1 

ri 

78.5 

76.5 

68.0 

56 . 3 

-*» 

54. 4 

rn  -> 

.  1  •  >  • 

M 

7.8 

14.8 

47.6 

17.8 

40.  2 

72.  1 

16.1 

0 

66.  1 

49.8 

60.3 

55.7 

58.3 

43.  1 

64.9 

p 

!  55.2 

59.  3 

60.  8 

54.2 

48.0 

90.  1 

59.7 

3 

71.8 

63.2 

67.8 

63.  1 

58.3 

59.  3 

67 . 6 

T 

!  59.7 

34.6 

61.1 

39.  5 

47.4 

71.3 

53.3 

U 

32.  1 

47.  3 

77. 3 

42.  4 

33.3 

3o.  1 

44.6 

1  « 

V 

|  31.6 

45.5 

75 .  5 

46.3 

65.3 

nn  n 

7  *1.  •  a- 

r~  -*»  rr 

.J  .  -J 
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$  *  $  CONCEPTUAL  LEVEL  *  »  t 

CONCEPT  -  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  3VJ1EM  « 


EVALUAllON  UP  CONCEPTS. 


TASK  RECRUIT 


EXPERTS: - 

C ONCER  IS: - 


A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

M 

N 


tl6 

cu'Tiiuni 

DEIIl'GPA 

MI  LI TAR 

PERSONA 

BEHAVIO 

SPEC  IF  1 

OVERAL L 

1.0 

25.  1 

44.0 

6.  0 

1.0 

13.9 

8.  1 

ao .  •:» 

35.0 

75 . 0 

76.  1 

34.9 

39.3 

03.  1 

45.  1 

51 . 1 

31.6 

5 1 .  o 

52.4 

55.  1 

10.  2 

22 . 0 

32.  7 

*r  >6 
.M  •  _ 

77.  1 

6'J  •  1 

32.  1 

25.  L 

6<>.  1 

43.9 

41.6 

r  -r  -4 

13.9 

49.4 

S3.  ■  * 

22.9 

84.6 

22.0 

34.  1 

29.0 

26.2 

4  3.0 

14.8 

69.  1 

70.9 

28.2 

57.3 

35.9 

36  •  3 

34.0 

48.6 

30 .  8 

39.7 

41.0 

45.4 

-39.5 

66.7 

58.2 

37.4 

33 .  tj 

45.7 

03.3 

6  *1 . 3 

48.6 

35.3 

30.  1 

53.9 

52.7 

17.  3 

39.  1 

8.9 

35.6 

59.5 

17.2 

13.  9 

24.8 

17.2 

64.5 

72.  1 

47.5 

73.6 

71.4 

/  >->  • 

70.  1 

56.5 

73.6 

35.  7 

69 .  5 

56.  7 

61.2 

56.3 

11.2 

1.2 

23.  1 

21 . 4 

2o  •  5 

34.  4 

1  .  'J 

53.9 

60.  1 

23.  1 

43.4 

59.4 

S3.  6 

56.3 

57.  1 

01.3 

2P.4 

96.3 

52.9 

99.0 

36 . 6 

41.2 

54.7 

94.3 

47.7 

71.4 

14.5 

3*1.  4 

50.3 

4  4 

53.  6 

*7.  4 

S5  •  6 

35.7 

47.9 

21.3 

54 . 1 

47.7 

43.  1 

29.4 

19.5 

'29.7 

Cl 

♦  ■l 

35.5 

35.  0 

29.  1 

71.9 

62.  2 

37. 0 
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*  *  CONCEPTUAL  LL'.TI  *  t  * 

CDNniPT  *  PROFILE  OP  THF  SUCCESSFUL  RECRUJ  tER 


EaRPRT  ’’./Lilt  tl 


!*-*“■  P  1  ' — o  i  30. *  ^  ■  i 

BFLOW  STANDARD 


40-4-- 

SUPERIOR 


AO—* _ 


9m 


T>'*SK  •-  RECRUIT 


EVALUATION  (JF  CONCEPTS 


EXPERTS 
CONCEF TS 

A 

B 

r; 

D 

F 

F 

G 

H 

I 

J 

F 

L 

M 

M 

0 

P 

s 

T 

U 


— —  #MRA 

COMMON  r  DENOGRA 


1.0 

1 

1 

>  J  1 

| 

•  j 

h*  | 

?  *1  •  3 

75. 6 

54. S 

58.6 

36 . 0 

29.  4 

72.2 

67.8 

31.6 

So.  y 

24  ^ 

SO.  7 

40.  2 

51.8 

74.6 

68.  5 

51.3 

34.  1 

6.5 

34  „  <S 

77.  a 

74.0 

80.5 

86.  2 

1 . 0 

1 . 0 

60.  5 

55.  7 

54.  u 

69 . 4 

63.  9 

73.8 

56.  5 

*T  .1  er 

21.2 

43.8 

25.  4 

37.  1 

MIL.  I  TAR  PERSONA  BEHAVIO  SFECIFI 


66 . 5 

1.0 

1.0 

-r 

/  /  .  „• 

67.5 

27.  S 

70.  3 

48.  2 

44.  I 

80.8 

30.5 

IS.  7 

64.8 

S3.  3 

^  r.t  n 

75.3 

23  •  5 

27.5 

77 .  a 

17.4 

"T*7  '-» 

68.3 

37.4 

37.0 

64,  4 

54.7 

73.3 

42.  7 

4  1.1 

62.3 

8.  7 

9.  3 

85.  1 

62 . 6 

44.8 

74.6 

46.  1  ' 

26.4 

67.  7 

11.0 

1 . 0 

140.  O 

S3.  7 

40.  9 

82.6 

50.3 

30.9 

7  1  •  & 

58.  9 

*41.2 

79.  7 

17.  9 

29.6 

73.  8 

10.7 

■77  ^ 

78. a 

57.5 

17.7 

7.  a 

79.  a 

63.0 
3a.  s 
IS.  6 
13.  4 
SS .  4 

-7  •  .J 

64.9 
37.  4 

r>cr  ir- 

«  .  j 

23.  1 

64. 2 
67.  u 
6S.  0 

42.3 

63.9 
14.2 

t — •  . 

J 


OVFRA  t.l 

1.0 

11.0 

4h.  : 


6o..,i 

76.0 

l  1.  1 

4 

4  7 .  '? 
40.  5 
i .  •  :> 
49.  .7 

42.  a 
1.0 
4  7 .  S 


40.3 
40.  3 

—  9 .  S 
15.<> 


172 


o  u  u.  u 


i 


*  *  «  CONCEPTUAL  LEVEL  *  1  t 

CONCEPT  -*  PROFILE  OF  THE  SUCCESSFUL  RECRUITER  EXPERT  St  IT  CM  ■* 


EVALUATION  OF  CONCEPTS 


T05I  =  RECRUIT 


EvPfiR  T3: -  #PPA 

CONCEPTS: -  LuMM'JN  I  OEMOGRA  MIL  I  TAR  PERSONA  BEHAVIO  SPECIF  I  OVERALL 


A 

U 

r 


H 

I 
J 
I . 
L 
M 
ri 
o 

p 

s 

T 

U 

V 


1.0 

-» r  t 

.  J  .  ' 

CD 

• 

fs 

**■ 

1.5 

1.0 

3.3 

1  . 0 

ai.o 

4i».2 

3 7.  j 

67.  1 

75.6 

85.8 

7  1 . 2 

46.4 

37.0 

c  r  -. 
00.0 

46.  9 

40.  4 

r  n  -• 

•-* 

1 

32.7 

33.0 

62.3 

48.  4 

23.3 

34.6 

4  2.2 

54.0 

64.2 

44.  a 

3 ...  6 

0  0.4 

23.3 

53.3 

23.7 

41.2 

o  — * .  9 

-» -* 

37.0 

30. 9 

37.  1 

22.6 

57.  7 

66.  5 

19.5 

31.9 

46.7 

.  - .  •  - 

35 . 3 

34.9 

31.3 

37.9 

33.3 

42.4 

41.2 

64.7 

36.2 

60.  a 

41.1 

57.6 

51.0 

60.  4 

49.4 

43.  0 

53.  B 

53.  9 

5-1.7 

50.  3 

59.2 

1  1 .  <t 

43.9 

41.3 

15.  2 

13.9 

20.4 

lO.  3 

65.5 

34.5 

72.6 

6<>.  1 

64.7 

69.6 

60.6 

66.0 

58.9 

69.0 

67 . 0 

43.6 

31.6 

64.-7 

er-  *1, 

J  i  k 

14.6 

47.6 

1.0 

24.8 

5...  9 

4.5 

5'.'.  4 

33.  7 

60.3 

47.8 

46.  3 

47.  1 

56.  1 

47.6 

33 . 5 

60.8 

49.  3 

33.  7 

50.3 

51.3 

46.  7 

5V. ') 

67.0 

C~  tr  ~y 

OO.  / 

59.  1 

47.3 

59.-7 

49.6 

46.7 

61.1 

23.  7 

74.3 

50.0 

42.7 

-Jo .  H 

47.  6 

77.  r 

33.9 

•TO.  1 

17.6 

40.  9 

74 . 3 

J 

•rr  r 
^0.0 

46.5 

42.6 

68.  1 

50 . 2 
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No.  Copios 


1.  Defense  Technical  Information  Center  2 

Cameron  Station 

Alexandria,  Virginia  22304-6145 

2.  Library,  Code  0142  2 

Naval  Postgraduate  School 

Monterey,  California  93943-5002 

3.  Professor  George  W.  Thomas,  Code  54Te  4 

Department  of  Administrative  Sciences 

Naval  Postgraduate  School 
- Monterey,  California  93943-5000 

4.  Professor  Stephen  L.  Mehay,  Code  54Mp  1 

Department  of  Administrative  Sciences 

Naval  Postgraduate  School 
Monterey,  California  93943-5000 

5.  Chairman,  Code  54  1 

Department  of  Administrative  Sciences 

Naval  Postgraduate  School 
Monterey,  California  93943-5000 

6.  Dr.  Paul  J.  Hoffman  1 

Magic  Seven  . . . 

101  First  Street,  Suite  237 

Los  Altos,  California  94022 

7.  Dr.  Walter  C.  Borman  1 


Personnel  Decisions  Research  Institute 
43  Main  Street  S.E. 

Riverplace,  Suite  405 
Minneapolis,  Minnesota  55414 

8.  _  Commander  -  1 

Navy  Recruiting  Command 
Attn:  Carl  Kannapel 
4015  Wilson  Boulevard 
Arlington,  Virginia 

9.  LCDR  Joyce  Zellweger  1 

2725  Manorhaven  Court 

Alexandria,  Virginia  22306 
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10.  Kathy  Kocher,  Code  54 
Department  of  Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  California  93943-5000 

11.  Helen  Davis,  Code  54  1 

Departaent  of  Administrative  Sciences 

Naval  Postgraduate  School 
Monterey,  California  93943-5000 

12.  Dr.  Deirdre  J.  Knapp  1 

U.S.  Army  Research  Institute 

5001  Eisenhower  Avenue 
Alexandria,  Virginia  22333-5600 

13.  Department  of  the  Navy  1 

Chief  of  Naval  Operations  (OP-114) 

Attn:  LCDR  Crowell 
Washington,  D.C.  20350-2000 

14.  LT  Robin  R.  Gandolfo  2 

607  Crescent  Drive 

Elizabethton,  Tennessee  37643 
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